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ABSTRACT 


PROCUREMENT  OF  COMMERCIAL  OFF-THE-SHELF  COMPUTER  EQUIPMENT 
FOR  USE  IN  NAVY  COMMAND,  CONTROL,  COMMUNICATIONS  AND 
INTELLIGENCE  SYSTEMS  by  LCDR  Herbert  Yee,  USN,  93  pages 

This  study  Investigates  the  procurement  of  commercial  off- 
the-shelf  (COTS)  con^uters  for  use  in  Navy  Commeuid,  Control, 
Communications  euid  Intelligence  (C3I)  systems. 

The  procurement  policy  was  reviewed  from  1970  to  the  present 
to  determine  what  chcuiges  were  effected  to  improve  the 
procurement  process.  Then,  a  case  study  was  conducted  which 
analyzed  the  Flag  Data  Display  System  (FDDS)  and  Joint 
Operational  Tactical  System  I  (JOTS  I) .  The  two  systems 
were  conpared  to  determine  if  the  JOTS  I  procured  under  the 
new  policy  better  supported  the  operational  requirements  of 
the  tactical  commander  at  sea  than  the  FDDS. 

The  conclusions  revealed  that  the  DOD  procurement  process 
was  streamlined  for  the  better  and  the  use  of  COTS  shortened 
the  acquisition  cycle  and  reduced  research  auxd  developmental 
costs.  The  conparative  analysis  of  FDDS  and  JOTS  I 
demonstrated  that  JOTS  I  was  not  operationally  effective  and 
suitable  in  meeting  the  tactical  commander's  requirements. 

It  was  recommended  that  with  the  reshaping  of  the  military, 
further  studies  should  be  conducted  on  how  to  better  use 
Navy  C3I  systems  in  operations  other  thcui  war  to  include 
joint  operations. 
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CHAPTER  ONE 


INTRODUCTION 

Importance  ot  the  study 

A  coinmcmd,  control,  communications  amd  intelligence 
(C3I)  system  is  the  most  importeuit  meams  by  which  a  Navy 
tactical  ccmnnander  controls /executes  authority  over  his 
forces.  Effective  C3I  enadsles  the  tactical  commander  to 
adequately  visualize  the  disposition  of  his  forces,  address 
the  tactical  problem  in  proper  perspective,  amd  respond  with 
a  rational,  auid  intelligent  decision. 

The  tactical  commamder  afloat  requires  a  C3I  system 
that  can  assimilate  the  necessary  information  quickly  and 
accurately  which  then  enadsles  the  commander  to  make  a 
decision  auid  disseminate  orders  to  sxibordinate  commanders. 
Further,  the  quality  amd  capability  of  the  systems  must 
adhere  to  stringent  standards  to  ensure  the  commander 
receives  accurate  information.  A  C3I  system  containing 
deficiencies  can  severely  impair  the  tactical  commander’s 
decision  making  capability  and  possibly  be  detrimental  to 
the  mission.  To  ensure  the  commander  has  an  effective  C3I 
system,  the  Navy  has  established  a  program  to  evaluate  a 
system’s  operational  effectiveness  and  suitability  to 
support  the  tactical  commander  afloat.  The  Commander, 
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Operational  Testing  and  Evaluation  Force  (COMOPTEVFOR) ,  has 
the  responsibility  to  test  and  evaluate  C3I  systems  to 
determine  whether  standards  are  met. 

The  C3I  systems  presently  in  the  fleet  are  obsolete 
when  compared  to  the  complex  auid  sophisticated  weapons 
systems  that  generate  the  combat  power  required  to  destroy 
the  enemy.  Additionally,  advcuiced  technology  has  enabled 
the  military  to  build  sophisticated  intelligence  collecting 
devices  that  produce  an  inordinate  amount  of  tactical 
information  that  can  overwhelm  the  tactical  commander.  Yet, 
the  C3I  systems  that  gather,  process,  and  display  this 
information  are  not  of  the  same  caliber.  The  Navy  must 
narrow  this  technology  gap  between  C3I  systems  and  weapons 
systems  by  placing  greater  eit^Jhasis  on  developing  iirproved 
C3I  systems. 

In  the  present  state  of  budget  reductions,  the  Navy 
must  ensure  that  it  is  getting  the  best  for  its  money 
without  compromising  the  stringent  military  standards  in 
system  performance.  Because  the  C3I  system  represents  the 
tactical  commander's  nerve  center,  it  is  an  cQssolute 
necessity  that  the  commander's  C3I  system  has  the 
sophistication  to  manage  the  data. 

The  Statement  of  the  Problem 

This  research  proposed  to  evaluate  the  operational 
effectiveness  and  suitability  of  commercial  off-the-shelf 
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(COTS)  computer  equipment  acquired  under  the  new  procurement 
policy  for  nondevelopmental  items  (NDI)  to  meet  the  Navy's 
requirements  for  a  tactical  Command,  Control,  Communications 
and  Intelligence  (C3I)  system  afloat.  The  focus  of  the 
research  was  a  historical  analysis  of  procurement  policy  and 
a  con^arative  analysis  of  two  tactical  C3I  systems  afloat: 
one  procured  under  the  old  acquisition  policy  and  one 
procured  under  the  current  policy. 

The  Hypotheses 

1.  The  first  hypothesis  was  that  the  new  Department 
of  Defense  (DOD)  procurement  policy  is  better  than  the  old 
one  in  terms  of  streamlining  the  acquisition  cycle  auid 
reducing  associated  costs. 

2.  The  second  hypothesis  was  that  COTS  computer 
equipment  employed  in  a  tactical  environment  to  support  the 
Naval  Commander  afloat  meets  his  operational  requirements 
but  does  not  meet  the  standards  promulgated  in  the  Test  and 
Evaluation  Master  Plan. 

The  Suboroblems 

1.  The  first  subproblem  was  to  determine  the 
differences  in  DOD's  current  procurement  policy  in 
comparison  to  the  former. 

2 .  The  second  subproblem  was  to  determine  the 
operational  effectiveness  and  operational  suitability  (i.e. 
reliability,  availability,  and  maintainability)  of  the  COTS 
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computer  equipment  when  compared  to  a  system  designed  to 
meet  military  specifications  (mil- specs)  for  the  naval 
commeuider ' s  C3I  requirements  in  a  tactical  environment. 

Limitations 

This  study  addressed  two  types  of  C3I  systems 
installed  in  conventional  and  nuclear  powered  aircraft 
carriers  (CV  &  CVN) .  One  C3I  system  type  designed  to 
military  specifications  was  the  Flag  Data  Display  System 
(FDDS) ;  the  second  Joint  Operational  Tactical  System  (JOTS) 
used  a  commercial  desktop  computer. 

Both  systems  addressed  in  this  study  perform 
information  gathering,  processing,  display  euid  dissemination 
in  support  of  the  naval  commcuider  afloat. 

The  operational  evaluation  (OPEVAL)  reports  provided 
the  tool  to  compare  the  two  systems. 

Delimitations 

This  study  did  not  address  systems  integrated  with 
weapons  systems. 

This  study  did  not  address  C3I  systems  installed  - 
aboard  US  submarines  or  aircraft. 

This  study  did  not  address  C3I  systems  prior  to  1970. 

This  study  did  not  address  software  procurement 
policies . 

This  study  did  not  use  threat  assessment  reports 
(secret  level)  to  evaluate  the  threat (s). 
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The  study  did  not  address  the  different  modes  of 
communications  (i.e.,  radio,  satellite)  associated  with  C3I 
systei.  . . 


Definitions 

Commercial  Item  Description  (CIP) •  a  commercial  item 
description  is  a  specification  that  describes,  by  salient 
functional  or  performance  characteristics,  the  available, 
acceptable  commercial  or  commercial- type  products  that  will 
satisfy  government  needs.  It  is  a  type  of  federal 
specification.^ 

Carrier  Intelligence  Center  (CVIC) .  A  secure 
confer tmented  space  edsoard  a  conventional  or  nuclear 
aircraft  carrier  where  intelligence  information  is  gathered, 
processed  and  assimilated  for  battle  group  use. 

Combat  Direction  Center  (CPC) .  The  secure 
compartment  aboard  a  conventional  or  nuclear  aircraft 
carrier  from  which  the  commander  controls  the  defense  of  the 
aircraft  carrier. 

Command  and  Control  (C2J .  The  exercise  of  authority 
cuid  direction  by  a  properly  designated  commander  over 
assigned  forces  in  the  accoir^lishment  of  his  mission.  C2 
functions  are  performed  through  an  arrangement  of .  personnel, 
equipment,  communications,  facilities  and  procedures  which 
are  employed  by  a  commander  in  planning,  directing, 
coordinating,  and  controlling  forces  and  operations  in  the 
accomplishment  of  his  mission.* 
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Coromand.  Control.  Congnunications  and  Intelligence 


(C3I)  system.  Any  automated  computer  system  that  collects, 
fuses,  exchanges  data  and  displays  it  for  a  tactical 
commander  to  interpret  and  make  an  informed  decision. 

Commercial  off-the-shelf  (COTS) .  "Products  in 
regular  production  sold  in  substantial  quantities  to  the 
general  public  and/or  industry  at  established  market  or 
catalog  prices."* 

Electromagnetic  pulse  (BMP) .  "Current  and  voltage 
surges  triggered  by  a  nuclear  blast  above  the  earth's 
surface. "* 

Flag  Data  Display  System  (FDDS) .  A  C2  system 
procured  under  the  old  procurement  policy  and  installed  in 
the  Tactical  Flag  Commcind  Center  (TFCC)  ,  CDC,  CVIC  and 
Supplementary  Plot  (SUPPLOT)  of  both  conventional  and 
nuclear  aircraft  carriers  to  support  the  tactical  commander 
afloat . 

Joint  Operational  Tactical  System  (JOTS) .  An  interim 
C2  system  procured  under  the  new  procurement  policy  and 
installed  in  the  TFCC,  CDC,  CVIC  and  SUPPLOT  of  both 
conventional  and  nuclear  aircraft  carriers  to  support  the 
tactical  commander  afloat. 

Military  specification.  A  military  requirement  that 
a  weapons  system  must  be  able  to  perform  under  certain 
parameters . 
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computer  installed  onboard  combat  ships  and  some  aircraft 
that  enables  communicating  information  such  as  position, 
course,  speed  and  altitude  to  other  NTDS  equipped  units. 

Nondevelopment al  Item  (NDI) .  "To  include  items  that 
are  either  available  in  the  commercial  marketplace  or 
otherwise  already  developed  and  in  use  by  a  government 
entity  in  this  or  an  allied  country."* 

Operational  Evaluation  (OPEVAL) .  The  phase  of 
operational  testing  and  evaluation  in  which  all  known 
problems  and  deficiencies  in  developmental  testing  are 
corrected.  These  deficiencies  included  hardware  and 
software  related  problems  observed  in  the  engineering  and 
development  of  the  system.  Once  all  deficiencies  are 
corrected,  the  program  manager  certifies  a  system  as  ready 
for  OPEVAL.  Favorable  results  and  conclusions  lead  to  a 
milestone  III  decision  that  recommends  going  into  full 
production. 

Operational  testing  and  evaluation  (OT&E) .  "Testing 
and  evaluation  conducted  to  estimate  a  system's  operational 
effectiveness  and  operational  suitability,  identify  needed 
modifications,  and  provide  information  on  tactics,  doctrine, 
organization,  and  personnel  requirements."* 

Officer  in  Tactical  Command  Information  Exchange 
System  (OTCIXS) .  A  two-way  satellite  communications  system 
for  ashore  and  afloat  units  with  satellite  equipped 
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capability. 


Prototype  Ocean  surveillance  Terminal  (POST) .  A 
Hewlett  Packard  desktop  con^uter  developed  to  perform 
correlation  of  electronic  intelligence  (ELINT) .  it  was 
installed  as  a  subsystem  to  pre-process  BLINT  data  prior  to 
being  forwarded  into  FDDS,  Tomadiawk  Weapons  Control  System 
(TWCS)  or  JOTS. 

Supplementary  Plot  (SUPPLOT) .  A  secure 
compartmented  space  aboard  a  conventional  or  nuclear 
aircraft  carrier  in  which  special  intelligence  information 
is  routed  for  initial  processing.  This  was  one  of  four 
locations  in  which  a  FDDS  or  JOTS  terminal  was  installed. 

Tactical  environment.  Includes  both  peacetime  and 
wartime  environment.  Peacetime  environment  encoitpasses 
underway  steaming  under  nonhost ile  conditions.  Wartime 
environment  encompasses  underway  steaming  in  which  the 
threat  of  imminent  attack  by  hostile  forces  is  possible. 

Tactical  C3I  system.  "Those  C3I  systems  and 
equipment  which  are  developed  and  acquired  for  use  by 
tactical  forces."’ 

Tactical  Flag  Command  Center  (TFCC) .  The  secure 
compartment  in  a  conventional  or  nuclear  aircraft  carrier 
from  which  the  embarked  commander  and  his  staff  operated, 
and  where  they  made  tactical  decisions.  This  was  one  of 
four  locations  in  which  a  FDDS  or  JOTS  system  was  installed. 
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Test  and  Evaluation  Master  Plan  (TEMP) .  A  document 


that  provides  overall  guidance  in  the  procurement  of  a 
system.  The  document  typically  describes  the  equipment 
characteristics,  developmental  and  operational  testing 
schedules,  tentative  milestone  achievement  dates,  critical 
operational  issues  (test  objectives)  for  evaluation  and 
identifies  resources  required  to  conduct  the  testing. 

Then  Year  Dollars.  The  value  of  current  year  dollars 
from  years  past. 

Tomahawk  Weapons  Control  System  (TWCS) .  A  shipboard 
cruise  missile  weapons  system  that  included  an  integrated 
command  and  control  suite.  Its  C2  system  in  functionality 
was  very  similar  to  FDDS.  The  SPRUANCE  class  destroyers  cuid 
AEGIS  class  cruisers  en^loy  this  system. 

Weapons  system.  A  generic  term  to  describe  any  type 
of  equipment  (i.e.  gun  system,  missile  system,  etc.)  with 
the  capability  to  launch  a  missile  or  fire  a  projectile. 

Assumptions 

1.  The  operational  test  and  evaluation  criteria  were 
the  same  under  the  old  procurement  policy  as  in  the  new. 

2.  C3I  computer  equipment  was  tested  in  a  realistic 
operational  environment. 

3 .  The  computer  technology  used  in  the  design  phase 
was  the  state  of  the  art  for  that  era. 

4.  The  threat  discussed  in  the  National  Security 
Strategy  of  the  United  States*  and  Military  Strategy  Of  the 
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United  States*  accurately  reflect  the  threat  imposed  on  the 
country  and  dictates  what  weapons  systems  will  be  procured. 

5.  The  budget  constraints  in^osed  on  the  military 
have  required  the  Navy  to  change  its  method  of  weapons 
procurement . 

6.  Costs  were  either  "current"  or  "then  year" 
dollars. 
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CHAPTER  II 


LITERATURE  REVIEW 

Historical  Background 

In  1972  the  Commission  on  Government  Procurement 
introduced  the  recommendation  that  the  acquisition  process 
of  commercial  items  become  the  leading  policy  in  developing 
new  systems.  In  the  ensuing  years  the  DOD  increased  its 
awareness  of  using  NDI  and  en^hasized  the  need  to  follow 
this  path  of  prociirement  to  field  systems  more  rapidly. 

This  chamge  in  procurement  policy  did  not  occur  overnight. 
Rather,  it  was  an  evolutionary  process  in  which  study 
groups,  commissions  cuid  federal  agencies  studied,  and 
restudied  for  two  decades  before  producing  a  viable 
solution. 

In  the  1980 's,  the  timing  seemed  appropriate  in  the 
political  cuid  militaury  arenas  for  identifying  alternative 
mrthods  of  military  weapons  systems  procurement.  During 
President  Reagan's  tenure  he  ordered  two  studies,  one  in 
1983  by  the  Grace  commission  and  the  other  in  1986  by  the 
Packard  commission  (President's  Blue  Ribbon  Commission  on 
Defense  Mcinagement)  to  investigate  the  defense  acquisition 
system.^  The  Packard  commission's  report  was  the  most 
comprehensive  and  accurate  compared  to  those  con^leted  in 
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the  past.  The  commission  interviewed  many  experts  and 
critics  in  the  field  concerning  the  various  aspects  of  the 
current  accjuisition  process.  One  of  the  major 
recommendations  made  by  the  Packard  Commission  was  to  expand 
the  use  of  commercial  products  rather  theui  develop  items 
under  military  specifications.^ 

The  literature  reviewed  for  this  study  provided  good 
general  information  on  the  C3I  concept.  As  an  exan^le  a 
book  titled  C3I;  Issues  of  Command  and  Control,  consolidated 
many  excerpts  from  an  extensive  niimber  of  sources  to  provide 
cui  indepth  look  at  all  Command  auid  Control  issues.^  This 
book  ranged  from  a  very  broad  overview  to  specific  issues 
that  included  buying  COTS  items  in  order  to  shorten  the 
procurement  cycle.  It  also  provided  a  historical 
perspective  on  the  importance  of  C3I  to  commanders. 

Focusing  closer  on  the  thesis  topic  of  Navy  C2  systems, 
Kenneth  Allard's  book  command.  Control  and  the  Common 
Defense  provided  cui  excellent  source  of  historical 
information  cuid  also  discussed  the  TFCC  and  FDDS  installed 
onboard  aircraft  carriers.* 

NDI  Technology 

Loescher  in  "Navy  Reshapes,  Develops  Copernicus 
Architecture, "®  reviewed  and  discussed  the  latest 
innovations  and  advances  in  computer  technology  that  support 
the  development  of  improved  C3I  systems.  Other  authors* 
promoted  the  use  of  COTS  items  to  support  C3I.  The  testing 


13 


of  new  equipment  in  the  controlled  laboratory  environment 
showed  favoredjle  results;  however,  no  evidence  exists  that 
indicates  the  equipment  tested  would  prove  reliable  in  an 
operational  environment. 

An  examination  of  the  ideas  and  thoughts  expressed  by 
military  researchers^  suggested  that  the  lead  time  between 
conceptualization  and  development  was  too  long  for  systems 
procured  under  military  specifications  and  standards. 
Additionally,  insufficient  attention  was  given  to 
calculating  long-term  costs.  In  many  cases  the  final  costs 
were  much  higher  than  the  original  estimates.  The  recurring 
conclusions  indicated  that  buying  commercial  off-the-shelf 
(COTS)  items  saved  money  and  reduced  the  acquisition  cycle. 

Test  and  Evaluation  (T&B) 

The  Navy  required  that  all  systems  under  procurement 
undergo  rigorous  testing  and  evaluation  as  part  of  the 
acquisition  process.  Testing  was  a  time-consuming  auid 
costly  effort,  frequently  delaying  the  introduction  of  an 
item  needed  by  the  fleet.  A  review  of  the  Assistant 
Secretary  of  Defense's  docxoment  Buying  NDI*.  cui  Air  Force 
research  report*,  amd  aui  Army  Science  Board  study^“  on 
procuring  NDI  for  C3I  indicate  that  buying  NDI  reduced  the 
procurement  time.  The  consensus  was  that  if  the  vendor's 
testing  was  comparable  to  the  military  standards  of  testing, 
then  the  developmental  testing  was  waivable. 

Regardless  of  any  savings  gained  in  terms  of  money 


14 


and  time,  a  Navy  system  under  procurement  must  pass  its 
final  phase  of  testing  before  full  production  can  begin: 
operational  testing  and  evaluation  (OT&E) .  In  reviewing  the 
Navy's  operational  assessment  and  test  and  evaluation 
reports^^  on  C3I  systems  conducted  over  the  last  12  years,  I 
noted  the  reports  provided  detailed  background  information 
on  each  C3I  system  under  evaluation.  Additionally,  the 
reports  provided  a  chronology  on  the  evolution  of  the 
system;  insights  in  how  the  particular  system  evolved  ouid 
came  into  being;  a  forecast  of  the  project's  future  and 
program  direction;  and  the  follow-on  scheduled  milestones  in 
the  procurement  process. 

Above  all,  the  most  in^ortant  facets  of  OT&B  were  the 
published  results,  conclusions  and  recommendations  which 
determined  whether  the  Navy's  requirements  for  a  system  were 
satisfactorily  met  and  whether  the  system  was  ready  for 
introduction  into  the  fleet. 

Military  Specifications  (Mil -spec) 

When  discussing  COTS  items,  the  frequently  asked 
question  was  whether  cui  item  conformed  to  mil -spec  and/or 
military  standards  (mil-std) .  Mil- spec  is  of  great  concern 
and  draws  interest  in  the  military  euid  commercial  industry. 
The  military  uses  mil -spec  to  ensure  systems  can  withstand 
the  harsh  military  operating  environment.  From  an 
industrial  standpoint  contractors  equate  mil -spec  to  their 
commercial  item  description  (CID)  in  establishing  standards 
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for  equipment.  In  the  last  five  years,  this  topic  has  drawn 
significant  attention  in  several  professional  journals. “ 

The  feature  articles  agree  that  environmental  forces  pose 
the  greatest  danger  to  computers.  For  example,  David 
Schiff,  a  processor  design  engineer  and  Paul  Wilson,  a 
senior  development  engineer,  discussed  the  issue  of  mil -spec 
from  both  a  ccsnmercial  euid  military  perspective.^^  From  the 
industrial  standpoint,  the  environmental  factors  that 
influenced  computer  development  primarily  focused  on 
withstanding  the  shock  of  shlpnent  and  the  mild  fluctuations 
in  temperature  associated  in  transportation.  From  the 
military  perspective,  specifically  the  Navy,  fluctuations  in 
power  and  operations  in  extreme  ten^erature  ranges  appeared 
to  be  the  most  important  factors.  In  essence,  the  degree  of 
environmental  Influence  was  the  difference  between  mil -spec 
cUld  CID. 

Over  the  years,  improvements  in  circuitry  design  and 
technological  advances  have  led  to  more  sturdy  and  relicQsle 
desktop  con^uters.  In  a  30 -month  field  test  of  40  desktop 
computers  by  the  lOth  Mountain  Division,  the  computers  were 
subjected  to  field  operating  conditions  in  Puerto  Rico  and 
Germany  in  temperatures  ranging  from  -40  degrees  fahrenheit 
to  +100  degrees  fcQirenheit  with  only  two  minor  failures.^* 
This  example  illustrates  that  CID's  in  which  commercial 
con5)uters  are  built  have  come  closer  to  mil -specs  but  have 
not  yet  and  probably  never  will  accommodate  the  stringent 
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requirements  for  military. 

As  the  DOD  saw  the  utility  of  using  CID's  over  mil- 

spec,  CID's  were  favored  over  mil-specs  in  designing 

systems.  In  the  Secretary  of  Defense's  (SECDEF)  1992  annual 

report  to  the  President  and  Congress,  he  stated  that. 

Over  35,000  military  specifications  and  standards 
were  reviewed  with  the  intent  of  replacing  as  many 
as  possible  with  commercial  item  descriptions. 

As  a  result,  2,500  have  been  canceled  outright,  1,500 
have  been  replaced  with  CID's  or  industry  steuidards.^^ 

As  the  military  budget  continues  to  decrease,  DOD  will  place 

greater  en5)hasis  on  using  CID's  in  lieu  of  mil -specs.  The 

military  will  eliminate  much  developmental  testing  providing 

that  the  vendors'  testing  methodology  is  acceptcdsle.  The 

result  will  be  a  reduction  in  overall  R&D  costs  and  time. 

The  Ihrsat 

A  major  consideration  in  determining  the  requirements 

of  a  weapons  system  is  the  threat.  At  the  strategic  level, 

documents  signed  by  the  President,  Secretary  of  Defense  and 

the  Chairman,  Joint  Chiefs  of  Staff  (JCS)  indicate  that 

The  Soviet  threat  which  once  faced  the  United  States 
has  virtually  diminished.  In  fact  it  is  difficult 
to  identify  one  threat;  cuid  conceivable  that  there 
are  multiple  threats  because  of  the  changes  in 
international  security  throughout  the  world. 

The  Chairtticui,  JCS  does  attempt  to  be  more  specific  and 

points  out  that,  "North  Korea,  a  weakened  Iraq  and  a  hostile 

Irani  pose  a  threat  aihd  thus  require  US  military  forces  to  be 

maintained. 

The  US  Navy  has  reassessed  its  role  in  the  military 
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and  published  a  new  policy  in  view  of  the  on-going  changes 
in  the  world.  According  to  the  Secretary  of  the  Navy's 
White  Paper/*  mines,  sea -skimming  cruise  missiles,  cuid 
tactical  ballistic  missiles  remain  a  formidable  threat 
against  the  Navy's  current  systems  and  force  structure.  If 
this  in  fact  is  the  case,  then  the  Naval  commander  afloat 
must  have  the  capability  to  rapidly  evaluate  a  possible 
hostile  threat  that  possesses  the  capedaility  of  launching  a 
low  altitude,  high  speed  weapon  with  a  small  cross  sectional 
radar  image  or  a  long  range  high  altitude  missile.  In 
either  case  the  Naval  commander  must  react  decisively  to 
counter  the  threat. 


Survivability 

As  important  as  mil -specs  amd  mil-stds  are  in  the 
technical  design  of  C3I  coit^uter  equipment,  the  end-users 
wemt  equipment  that  is  survivable  in  am  operational 
environment.  Professional  writers  present  a  broad  rauige  of 
opinions  and  comments  on  survivability.^*  Survivability 
can  be  viewed  from  two  perspectives,  a  peacetime  amd  wartime 
environment.  Perhaps  the  most  common  and  prevalent  question 
is  whether  C3I  equipment  cam  survive  in  a  worst  case 
scenario,  the  effects  of  electromagnetic  pulse  (BMP)  from  a 
nuclear  attack  in  a  wartime  environment.  Major  Melvin  Hoke, 
an  Air  Force  officer  assigned  as  an  electromagnetic 
compatibility  engineer,  discusses  the  use  of  NDI  in  a 
nuclear  environment.^*  In  a  peacetime  environment 
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survivability  in  terms  of  the  operating  environment  include 
extreme  temperature  changes  and  power  fluctuations. 

Conclusion 

The  literature  review  has  highlighted  some  of  the 
significant  areas  that  relate  to  the  subproblems  stated  in 
chapter  I,  and  in  turn  the  primary  thesis  question.  By 
euialyzing  the  procurement  process  over  the  past  two  decades, 
the  use  of  NDI  technology  over  military  specifications,  and 
the  test  euid  evaluation  process,  I  atten^ted  to  assess  if 
the  military  has  taken  the  appropriate  and  necessary  steps 
to  reduce  costs  and  streamline  weapons  system  acquisition  by 
applying  COTS  over  mil -spec  items  in  developing  new  C2 
systems  with  respect  to  meeting  the  operational  requirements 
of  the  user.  However,  literature  regarding  system 
survivability  in  view  of  a  nuclear  threat  was  limited  and 
the  threat  of  computer  viruses  to  software  was  not  existent . 
Additionally,  with  the  recent  use  of  COTS  equipment,  the 
intact  of  the  tradeoffs  in  system  performance  and 
maintenance  have  yet  to  be  determined  in  tezms  of  affecting 
a  tactical  commander's  decision-making. 
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CHAPTER  THREE 


METHODOLOGY 

This  research  used  two  methodologies  to  euiswer  the 
Investigative  questions  pertaining  to  the  research  question. 
I  used  the  historical  method  to  euiswer  the  first  subproblem 
and  the  case  study  approach  to  answer  the  second. 

Subproblem  Number  l 

The  first  subproblem  was  to  determine  the  differences 
In  current  DOD  procurement  policy  In  conparlson  to  the 
previous  one. 

1.  The  data  needed.  The  data  needed  for  solving  the 
first  sTibproblem  were  national  level  dociaments.  Secretary  of 
Defense  reports,  DOD  Directives,  JCS  publications,  SECNAV 
Instructions,  and  government  studies  pertaining  to  the 
acquisition  policy  for  commercial  off-the-shelf  Items. 

2.  The  Location  of  the  Data.  The  Combined  Arms 
Research  Library  (CARL)  contained  all  data  with  the 
exception  of  SECNAV  Instructions.  SECNAV  Instructions  were 
found  In  the  Navy  office  In  Bell  Hall.  All  data  was  readily 
available. 
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3.  Treatment  of  the  data. 

a.  Analysis.  I  used  these  reports  and  documents 
were  used  to  identify  the  changes  and  revisions  in  the 
procurement  policy.  First,  I  reviewed  the  administration's 
policies  to  look  at  why  DOD  conducted  these  studies.  I  only 
examined  the  period  from  1970  to  the  present.  Secondly,  I 
reviewed  the  government  studies  conducted  to  identify  what 
were  their  findings  and  recommendations  regarding  the 
acquisition  policy.  Next  I  reviewed  the  SBCDBF  reports  to 
determine  what  actions  DOD  took  to  incorporate  those 
recommendations  into  DOD  Directives  and  Instructions  to 
support  in^rovements  in  the  acquisition  process.  And 
finally,  I  reviewed  the  monographs,  dissertations,  and 
journal  articles  to  determine  any  other  findings  used  to 
support  the  use  of  COTS  items  in  the  procurement  of  C3I 
systems . 

b.  Interpretation.  Analysis  of  the  literature 
provided  a  chronology  of  how  NDI  procurement  evolved,  and 
what  specific  changes  occurred,  that  led  to  the  conclusion 
of  whether  the  current  procurement  policy  is  better  or  worse 
than  the  previously  existing  one.  This  also  provided  a 
basis  for  answering  subproblem  number  2. 

Siibproblem  Number  2 

The  second  subproblem  was  to  determine  the 
operational  suitability  and  operational  effectiveness  (i.e., 
reliability,  availability,  maintainability,  etc.)  of  COTS 
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computer  equipment  to  meet  the  Navy's  requirements  in  a 
tactical  environment. 

1.  The  Data  Needed.  The  data  needed  to  resolve 
subproblem  #3  were  Operational  Test  and  Evaluation  (OT&E) 
Reports,  Test  cuid  Evaluation  Master  Plans  (TEMPs) , 
government  studies  and  articles  that  discussed  the  OT&E 
process . 

2.  The  Location  of  the  Data.  The  OT&E  reports  were 
located  at  COMOPTEVFOR,  Norfolk,  VA.  Articles  and  writings 
pertinent  to  OT&E  were  found  in  periodicals  located  in  CARL. 

3.  The  Mecuis  of  Obtaining  the  Data.  The  CARL  through 
the  Defense  Technical  Information  Center  system  secured  the 
required  data. 

4.  Treatment  of  the  Data. 

a.  Analysis.  The  technique  I  used  was  the  case 
study  research  method  to  review  the  data  and  compare  Navy 
operational  test  and  evaluation  reports  for  C3I  systems 
procured  imder  the  past  «md  present  procurement  policy.  I 
then  cuialyzed  the  results  in  terms  of  the  operational 
effectiveness  and  suitability  of  the  systems  to  support  the 
tactical  commander.  Specifically,  I: 

(1)  Reviewed  and  analyzed  the  history  of  TFCC 
and  JOTS  programs  focusing  on  when  notable  events  occurred 
that  either  contributed  or  detracted  from  the  length  of  the 
procurement  time  and  the  associated  costs. 

(2)  Reviewed  journal  articles,  government 
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studies  and  DOD  Directive  5000.3  Test  and  Evaluation 
governing  the  OT&E  process.  Then  I  reviewed  the  test 
procedures  for  FDDS  and  JOTS  to  determine  if  they  conformed 
to  the  DOD  Directive  5000.3  Test  and  Evaluation. 

(3)  Reviewed  and  conqpared  the  test  results  for 
operational  effectiveness  in  the  OT&E  reports.  The 
principle  objective  was  to  determine  if  the  two  systems 
supported  the  tactical  ccxnmander  and  which  one  was  better  at 
providing  him  with  the  necessary  intelligence  to  make 
informed  decisions.  I  divided  operational  effectiveness  of 
the  C2  systems  into  elements  that  included  afloat  commander 
support,  information  exchange/processing,  data  base 
mcuiagement,  euid  correlation/matching.  To  ensure  a  fair 
evaluation  I  used  the  same  scale  to  cuialyze  functions  common 
to  both  systems.  “When  functions  were  available  in  one 
system  but  not  the  other,  I  evaluated  them  to  determine 
whether  they  enhcuiced  the  commander’s  C2  capability. 

(4)  I  reviewed  the  system  test  results  for 
operational  suitability  to  determine  if  the  thresholds  for 
reliability  (MTBF) ,  availability  (A(o) )  and  maintainability 
satisfactorily  met  the  old  emd  new  procurement  policies. 
Additional  suitedjility  issues  evaluated  were  survivability, 
logistic  supportability,  compatibility,  interoperability, 
training,  human  factors,  safety,  documentation,  and 
vulnerability. 
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b.  Interpretation.  In  the  conparison  of  the  data 
I  attempted  to  explain  how  one  C3I  system  procured  under  one 
policy  was  better  th2m  the  other  procured  under  the  other 
policy.  The  criteria  I  used  was  whether  it  met  the 
operational  requirements  of  the  tactical  commcuider. 
Additionally,  I  explained  how  one  policy  was  better  than  the 
other  in  terms  of  streamlining  the  acquisition  cycle  and 
reducing  associated  costs. 
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C3IAPTBR  POUR 


ANALYSIS 

Comnand  and  control  (C2)  systems  acquisition  is  one 
of  the  most  complex  challenges  in  militzury  weapon  system 
procurement.  C2  acquisition  is  Inherently  more  difficult  in 
that  the  requirements  are  less  definitive  emd  more 
conceptual  than  that  of  a  weapons  system,  which  is  designed 
with  a  specific  requirement  to  destroy  a  target.  The  lack 
of  threat  specificity  coupled  with  rapid  adveuices  in 
coK^uter  technology  has  made  C2  acquisition  even  more 
complex.  To  adhere  to  the  DOD  acquisition  policy  was 
difficult.  The  research  conducted  to  answer  the  thesis 
question  first  analyzed  the  procturement  process  and  then 
compared  the  operational  testing  euid  evaluation  results  of 
two  C2  systems  procured  under  different  policies. 

Siibproblem  Number  l 

The  first  subproblem  was  to  determine  the  differences 
in  the  Department  of  Defense's  (DOD)  current  acquisition 
policy  and  the  old  one.  To  answer  the  question,  I  analyzed 
acquisition  and  procurement  of  military  weapons  systems  from 
1970  to  the  present.  I  looked  at  DOD  policy  covering 
weapons  systems  acquisition  in  general  euid  then  focused  in 
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on  two  Navy  C2  systems.  Within  the  context  of  C2  systems,  I 
discussed  how  the  Navy  handled  the  challenges  of  procurement 
of  these  C2  systems.  To  adequately  address  the  first 
subproblem,  the  thesis  analyzed  government  studies, 
technological  advances  that  affected  acquisition  strategy, 
the  goal  for  achieving  greater  efficiency,  the  intact  of 
budget  reductions  and  the  nature  of  the  threat  during  the 
period  the  system  was  \uider  acquisition. 

Hiatory 

Fitzhugh  Commission 

For  decades,  the  OOD  has  had  the  luxury  of  operating 
relatively  Independent  with  little  intervention  by  outside 
orgeuiizations  or  agencies  in  regard  to  how  business  should 
be  conducted  or  run.  in  July  1969,  President  Nixon  and 
Secretary  of  Defense  (SECDEF)  Melvin  Laird  gave  the  Fitzhugh 
Commission  (President's  Blue  Ribbon  Defense  Panel)  the  task 
of  conducting  an  indepth  study,  of  the  Department  of 
Defense's  (DOD)  orgeuiization  and  management  practices  and 
ntcUcing  recommendations. 

After  a  year  long  study  and  examining  procurement 

policies  and  procedures  the  Fitzhugh  Commission  reported  to 

the  President  in  July  1970  that. 

The  acquisition  of  weapons  systems  and  other  hardware 
had  contributed  to  serious  cost  overruns,  scheduled 
slippages  and  performance  deficiencies.  Furthermore, 
the  panel  observed  that  the  difficulties  did  not  appear 
amenable  to  a  few  simple  cure-alls,  but  required  many 
interrelated  changes  in  organization  and  procedures.^ 
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The  commission  identified  two  key  areas  requiring 
change.  First,  the  panel  found  that  deficiencies  in  the 
operational  testing  and  evaluation  (OT&B)  process  was 
infrequent,  poorly  designed,  and  generally  inadequate  in 
measuring  the  effectiveness  of  weapons  systems  prior  to 
fielding.^  It  recommended  that  a  Defense  Test  Agency  be 
established  xinder  the  supervision  of  a  civilian  director 
directly  responsible  to  SBCDEF  to  mcuiage  DOD  testing  and 
evaluation.  Under  the  policy  then,  in  effect,  the  services 
conducted  their  own  testing  and  evaluation  and  reported  to 
their  respective  service  chiefs.  No  one  individual,  either 
military  or  civilian,  maintained  oversight  of  the  OT&B 
process . 

Secondly  the  Fitzhugh  Commission  identified 

deficiencies  in  research  and  development.  Specifically, 

unproven  technological  advances  produced  risks  and 

uncertainty  in  the  development  of  systems.  This  resulted  in 

cost  growths,  delays  in  the  program,  and  shortfalls  in 

desired  performance  of  the  system. 

In  March  1971,  DOD- selected  procurement  reports  of 
45  systems  amounted  to  $110  billion  cuid  accotinted 
for  cost  overgrowth  in  the  following  categories: 
technical  changes,  20%;  delivery  schedule  changes, 

17%;  abnormal  economic  fluctuations,  18%; 
incorrect  estimates,  29%;  and  other  causes,  16%.^ 

The  Fitzhugh  Commission  also  found  that,  "Cost  overruns  were 

not  new  but  in  the  1960 's  and  1970 's,  they  attracted  public 

awareness  to  an  extent  uncharacteristic  of  previous  times."* 

Subsequently  the  American  public  demanded  increased 
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accountcUaility  for  the  military  to  ensure  tax  dollars  were 
being  spent  wisely. 

The  commission  also  found  that  mcuiagement 
deficiencies  were  a  substantial  contributor  to  the 
acquisition  problems.  For  exan5>le,  one  of  the  problems  was 
Robert  McNamara's  failure  to  reorganize  the  DOD  to  emulate  a 
large  private  corporation  and  achieve  success  in  the 
1960 's.*  Another  major  problem  noted  was  that  frequent 
officer  rotations  did  not  allow  the  officers  the  time  to 
gain  the  necessary  experience  to  perform  adequately  at  their 
jobs  until  near  the  con^letion  of  their  tours.*  In  a 
technical  area,  such  as  acquisition,  the  officer's  job 
assignment  needed  to  be  of  sufficient  duration  so  that  he 
could  become  thoroughly  involved  in  the  work  and  be  fully 
responsible  for  the  results.'^ 

The  Pitzhugh  Commission  recommended  that  DOD 
establish  career  paths  for  officers  assigned  to  staff, 
technical,  and  professional  fields  in  such  areas  as 
research,  development,  and  procurement.  Further,  the 
commission  recommended  that  the  duration  of  those 
assignments  be  increased  and  that  officers'  opportunities 
for  promotion  should  not  be  hindered  due  to  increased  tour 
lengths  in  those  specialty  fields.* 

DOD  received  the  Fitzhugh  Commission's 
recommendations  and  incorporated  several  of  the 
recommendations  to  include  extended  tour  lengths,  specialist 
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career  pathing,  and  establishment  of  an  additional  DOD  staff 
position.  In  SECDEP  Laird's  statement  before  the  Senate 
Armed  Services  Committee  in  February  1972,  he  announced  the 
appointment  of  a  Deputy  Director  for  Test  and  Evaluation  to 
improve  the  quality  of  testing.*  SECDEF  Lair'^  felt  this 
would  resolve  the  OT&E  issue. 

In  the  area  of  systems  acquisition  management,  SECDEF 
Laird  relayed  three  key  points  that  his  Deputy,  Dave  Packard 
observed  during  his  tenure.  First,  wrong  decisions  were 
made  to  start  new  but  xinrealistic  programs.  "Many  programs 
had  problems  because  they  were  poorly  defined  from  the 
beginning.  Frequently  more  performance  was  requested  than 
was  really  needed  in  a  new  weapon. To  correct  this 
deficiency  the  administration  provided  specific  guidance 
that  supported  national  security  objectives  in  procurement 
of  new  systems.  Additionally,  procurement  officials  began 
to  place  greater  eit^hasis  on  the  views  of  the  Joint  Chiefs 
of  Staff  euid  the  requirements  of  the  Unified  Commanders  in 
Chief.  Siibsequently  fewer  projects  were  cancelled. 

However,  almost  a  decade  elapsed  before  the  CiNC's  became 
intimately  involved  in  the  planning  of  C3I  requirements  and 
future  needs. 

Second,  Deputy  SECDEF  Packard  observed  that  DOD 
officials  accepted  cost  estimates  even  when  they  were 
unrealistic.  SECDEF  Laird  pointed  out  that  over-optimism  in 
evaluation  of  the  technological  difficulties  involved  in 
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achieving  desired  performance  confounded  the  problem  as  well 
as  the  under -estimating  of  the  time  and  costs.  These 
accepted  practices  contributed  to  the  problem. “ 

Third,  DOD  management  provided  ineffective  oversight 
in  procurement.  The  Pitzhugh  Conmission  recommended  the 
establishment  of  five  new  assistcuit  Secretaries  of  Defense 
for  iiif roving  DOD  management  and  organization.  However, 
SECDBF  Laird  opposed  the  reconsnendation  and  argued  that  he 
would  support  two  new  positions,  instead  of  five,  to  avoid 
increased  staffing.  He  believed  this  was  one  of  the 
inherent  problems  to  the  already  complex  bureaucracy  within 
the  DOD. 

At  the  middle  management  level,  SECDBF  Laird's 
solution  for  improving  mcuiagement  was  to  select  more  capcUsle 
project  managers  and  to  increase  effectiveness  by  keeping 
them  on  the  job  longer.  He  also  implemented  changes  to  the 
Defense  Weapons  Systems  Management  Center,  a  school  that 
provided  officers  entering  the  acquisition  field  the 
fundamentals  for  becoming  program  managers.  Those  changes 
would  ensure  that  managers  received  the  necessary  training 
to  perform  their  jobs  properly.” 

Although  DOD  established  standards  in  its  directives 
and  instructions,  the  services  did  not  adhere  to  them  when 
procuring  new  systems.  At  one  end  of  the  spectrxam, 
development  consisted  of  paper  studies  rather  than 
attempting  to  build  a  working  prototype  which  employed 
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highly  advanced  technologies.  The  result  was  the 
expenditure  of  funds  for  a  project  that  never  left  the 
drawing  board.  At  the  other  extreme,  full  production  began 
before  completion  of  the  system  research.^* 

SECDEF  Laird's  solution  to  solving  the  problem  of 
overlapping  production  schedules  with  those  systems  nearinc 
con5)letion  in  development  was  strict  adherence  to 
milestones.  Milestones  were  stages  in  a  program's 
acquisition  cycle  which  are  met  before  continuing  on  to  the 
next  phase  of  development.  To  achieve  a  specific  milestone 
a  system  under  procurement  had  to  con^lete  a  phase  of 
developmental  testing  and  operational  testing  amd  have  the 
required  documentation  before  continuing  on  to  the  next 
phase.  Additionally,  the  SECDEF  encouraged  the  use  of  a 
less  structured  approach  in  the  procurement  process. 

Program  managers  now  had  some  flexibility  in  "structuring 
development  contracts  to  provide  for  tradeoffs  between 
performance,  time  schedules,  and  costs  throughout  the 
development  of  the  program  until  the  weapon  was  approved  for 
production. 

Using  the  Fitzhugh  Consnission' s  report  as  an  impetus 
for  encouraging  changes  to  improve  the  DOD's  acquisition 
process,  SECDEF  James  Schlesinger  reported  to  Congress  in 
March  1974  his  outlooJc  towards  future  procurement  for  fiscal 
years  1975-1979.  His  approach  was  twofold.  First,  he 
en^hasized  the  need  to  gain  better  control  of  the  current 
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systems  under  procurement  to  preclude  further  recurrences  of 
the  same  problems  identified  by  the  Fitzhugh  Commission. 
Second,  he  was  going  to  take  a  more  sensible  approach  to 
buying  new  systems  so  that  future  systems  would  be  more 
cost-effective  and  provide  an  acceptable  level  of 
performance  at  an  affordable  price. 

To  gain  greater  control  of  those  systems  under 
procurement,  SBCDEF  Schlesinger  implemented  two  management 
policies.^’  For  those  programs  at  the  design  stage,  he 
promoted  greater  use  of  prototyping.  Prototyping  was  a  low 
cost  investment  to  determine  if  a  concept  was  actually  worth 
pursuing  before  the  commitment  of  significant  funds.  For 
those  programs  at  the  developmental  stage,  he  mandated 
initial  OT&E  before  production  could  begin.  This  effort 
reduced  the  need  for  costly  modification  and  increased  the 
probability  of  avoiding  failures  before  actual  production. 
Also,  by  establishing  a  Deputy  Secretary  of  Defense  for 
Research  and  Engineering  for  Test  and  Evaluation,  he  ensured 
enforcement  of  his  "Fly  before  Buy"  program  was  enforced. 

For  the  procurement  of  future  systems  based  on 
emerging  operational  requirements  (OR) ,  SEODEF  Schlesinger 
implemented  a  new  concept  of  "Design- to-a-Cost"  program  in 
fulfilling  an  OR  as  economically  as  possible.^*  His 
"Design- to-a-Cost"  concept  hinged  on  the  services  producing 
accurate  cost  estimates  for  new  systems.  Concurrent  to  the 
services  calculating  cost  estimates,  a  Cost  Analysis 
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Iir^rovement  Group  within  the  Office  of  the  Secretary  of 
Defense  (OSD)  developed  its  own  impartial  cost  analysis  of 
the  service's  proposed  system.  Then,  a  cost  conparison  was 
made  to  determine  the  reasonableness  of  the  system. 

Finally,  performeuice  characteristics  that  would  satisfy  the 
OR  and  a  tin^  schedule  were  developed  to  fit  the  estimated 
cost.  This  concept  also  allowed  program  managers  to  adjust 
the  perfonn2mce  characteristics  auid  time  schedules  as 
necessary  to  remain  in  line  with  costs. 

SECDEF  Schlesinger ' s  new  approach  to  weapons  systems 
procurement  also  fostered  the  use  of  commercial  steuidards, 
where  practical,  in  lieu  of  military  specifications  (mil- 
specs)  and  promoted  the  use  of  "off-the-shelf”  components. 

By  encouraging  such  tradeoffs,  he  allowed  maximum 
flexibility  in  how  a  system  was  to  be  built  which  realized 
savings  in  life-cycle  costs. This  issue  would  resurface 
later  in  the  Grace  euid  Packard  Commissions'  studies 
conducted  in  1982  and  1986.  Subsequently,  the  DOD  made  this 
procurement  method  official  policy,  and  it  became  the  norm 
for  all  services. 

Defense  Science  Board  (1977) 

In  December  1977,  the  Under  Secretary,  Defense 
Research  and  Engineering,  William  J.  Perry  established  the 
Defense  Science  Board  (DSB)  Task  Force  to  study  C2  systems 
management  cuid  make  recommendations  for  improvement.*"  The 
DSB,  chaired  by  Dr.  Solomon  Buchsbaum  examined  five  areas  in 
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command  and  control.  Three  areas  pertained  to  joint  and 
multi -service  issues  which  included  the  procurement  of 
systems  for  joint  and  multi-service  use;  the  effectiveness 
of  C2  procedures  in  a  joint  environment  such  close  air 
support  cuid  battlefield  interdiction;  and  the  relationship 
between  the  joint  interoperability  council  and  World  Wide 
Military  Command  emd  Control  System  council.  The  last  two 
areas  were  within  the  scope  of  this  thesis.  The  first  area 
examined  the  existing  procedures  and  directives  regulating 
weapons  systems  procurement  and  their  applicability  to  C3I 
systems;  and,  second,  the  existing  management  organization 
handling  C3I  systems  cuid  whether  or  not  changes  were 
necessary.  If  changes  were  needed,  "the  task  force  was 
urged  to  develop  recommendations  that  if  implemented,  would 
help  improve  the  design,  acquisition,  operation,  and 
evolution  of  command  and  control  systems."*^ 

The  DSB,  in  its  10  month  study  discovered  many  flaws 
in  the  existing  program  structure.  The  number  one  concern 
was  that  existing  C2  systems  were  not  commensurate  in 
technology  when  compared  to  the  modem  weapons  systems  of 
destruction.  In  the  past  the  services  placed  emphasis  on 
developing  destructive  weapons  systems  which  enabled  the 
United  States  to  maintain  the  edge  over  the  Soviet  Union, 
and  consequently,  C2  systems  development  received  a  much 
lower  priority. 

Second,  the  DSB  found  that  C2  systems,  unlike  weapons 
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of  destruction,  were  very  con^lex  in  design  and  had  unique 
characteristics.”  Development  was  primarily  evolutionary 
rather  than  developmental  in  nature.  As  new  advances  in 
technology  appeared,  they  became  part  of  existing  systems. 
The  primary  emphasis  of  this  effort  was  incremental  upgrades 
in  software  that  encd)led  increased  performance.  By  the  time 
a  single  system  entered  the  field,  its  characteristics  were 
very  much  different  from  what  the  original  design  called 
for.  This  uniqueness  ensured  C2  systems  did  not  follow  the 
standard  acquisition  path. 

The  DSB  concluded  that  DOD  Directive  5000.1,  Major 

Systems  Acquisition,  was  not  appliccible  to  the  procurement 

of  C2  systems.  The  DSB  strongly  recommended  an  addendum 

that  would  provide  the  necessary  guidance  to  program 

managers  for  C2  systems  acquisition.  The  DSB  also  proposed 

the  establishment  of  a  new  agency,  designated  as  the  Defense 

Command  and  Control  Systems  Support  Agency,  to: 

Assist  the  Unified  and  Specified  Ccxnmands  euid  the  JCS 
in  the  development  of  command  etnd  control  system 
requirements  and  specifications;  to  establish 
technical  standards  for  interfacing  specifications;  to 
perform  development  pleuming  including  alternative 
concept  trade-off  studies;  to  develop  master  pleuis  for 
programming  and  budgeting  of  various  C2  developments 
and  procurement.*’ 

The  DSB  also  produced  a  draft  addendum  (DOD  Directive 
5105. XX)  to  esteUslish  guidelines  for  what  the  Defense  C2 
Systems  Support  Agency  was  supposed  to  do.**  The  Under 
Secretary  urged  the  SECDEF  to  implement  the  board ' s 
recommendations  to  correct  those  deficiencies.  However,  DOD 
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never  incorporated  the  draft  addendum  as  a  supplementary 
directive  to  DOD  5000.1.  Instead,  the  DOD  published  it  as  a 
Defense  Acquisition  Circular,  which  did  not  have  the  same 
impact  as  a  DOD  Directive.*® 

DOD  Directive  5000.1  series  was  revised  and  updated 
in  1977  cuid  1980.  In  March  1982,  Deputy  SECDBP  Freuik 
Ceurlucci  made  the  first  significant  revision,  based  on  the 
Acquisition  In^rovement  Program  to  streamline  the 
acquisition  process.*®  The  major  chcuige  in  the  directive 
was  the  reduction  of  the  five  milestone  decision  points  to 
three,  thereby  taking  years  off  the  lifecycle  of  a  program. 

Grace  Consnission 

Defense  spending  was  a  priority  in  the  1970 's  in 
light  of  the  national  debt,  but  as  the  national  debt 
continued  to  climb  it  became  an  even  higher  priority.  "On 
June  30,  1982,  President  Reagan  signed  Executive  Order  12369 
and  established  the  President's  Private  Sector  Survey  on 
Cost  Control."**  The  45  member  task  force  committee  known 
as  the  Grace  Commission  included  chairmen  and  presidents  of 
leading  corporations  throughout  industry.  The  commission's 
objectives  focused  on  studying  DOD  meuiagement  practices  on 
weapons  procurement,  logistics,  financial  management, 
personnel,  legal,  and  legislative  issues.  The  commission's 
task  was  to  make  recommendations  to  increase  efficiency  and 
determine  ways  for  achieving  cost  savings.  The  commission's 
mandate  was  to: 
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Identify  opportunities  for  increased  efficiency  and 
reduced  costs  achievable  by  executive  action  or 
legislation,  and  specify  areas  where  further  study 
can  be  justified  by  potential  savings.^* 

The  Grace  ccatsnission  fouind  two  areas  that  required 
reform.  First,  the  commission  foxind  that  OSO  and  the  four 
branches  of  service  were  duplicating  their  efforts  in 
procurement  management.  The  OSO  through  the  Under  Secretary 
of  Defense  for  Reseeurch  and  Engineering  provided  guidance, 
policies,  2md  procedures  on  acquisition  of  weapons  systems 
in  accordance  with  the  DOD  Directive  5000.1  and  DOD 
Instruction  5000.2  (applicable  to  all  services) .^*  Bach 
branch  of  the  services  had  a  similar  orgeuiizational 
structure  in  which  program  managers  reported  to  their 
respective  service  secretary  and  then  the  service 
secretaries  reported  to  the  OSD.’°  As  identified  later  by 
the  Packard  Commission,  a  key  to  success  was  establishing 
the  shortest  lines  of  communications  possible  to  promote 
efficiency  and  get  the  job  done.’^ 

The  commission  reported  that  "DOD  should  initiate  a 
program  to  modernize  and  streamline  the  total  acquisition 
process  by  consolidating  the  entire  process  into  OSD.”^‘  As 
a  start,  it  recommended  that  "DOD  should  seek  legislation  to 
establish  the  position  of  Under  SECDEF  for  Accjuisition 
within  OSD."^^  The  Under  SECDEF  would  then  have  the 
authority  and  responsibility  to  develop  and  implement  a 
program  to  restructure  the  procurement  system. 

The  second  major  issue  identified  by  the  Grace 
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commission  was  the  lack  of  use  of  common  parts  and 

standards.  The  commission  found  that  each  service  carried 

out  procurement  independently  euid  in  areas  where  similar 

weapons  systems  were  under  acquisition,  they  duplicated 

efforts.  The  commission  reccmranended  that, 

DOD  should  mandate  the  use,  where  possible,  by  all 
services  of  common  hardware  components,  subsystems, 
equipment  and  other  parts  in  order  to  minimize 
acquisition  and  life- cycle  costs.  The  potential 
savings  would  equate  to  approximately  $2 . 3  billion 
annually.^* 

In  atten^ting  to  reduce  the  costs  of  components,  the 
commission  closely  scrutinized  the  use  of  mil -specs.  They 
discovered  that  the  application  of  mil -specs  was  often  over 
used.*® 

The  commission  found  that  "procurement  officials  were 
not  sufficiently  selective  in  choosing  only  the  particular 
military  specifications  that  were  truly  needed  in  relation 
to  the  end  item  being  procured."**  Program  nanager  often 
wrote  poor  requests  for  proposals.  Industry  officials 
believed  that  program  managers  concentrated  heavily  on 
determining  what  specifications  to  select  rather  them  the 
requirements  of  the  system.*’  As  a  result,  the  extensive 
time  and  effort  devoted  to  developmental  testing  of 
cort^onents  to  ensure  they  met  the  standards  specified  in  the 
contract  often  drove  up  costs. 
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Packard  Comnlssion 


In  July  1985,  President  Reagan  tasked  his  Blue  Ribbon 
CcHisnission  on  Defense  Mamagement  (Packard  Comnission)  to 
conduct  an  indepth  study  on  defense  management  and 
organization  and  report  its  findings  cuid  recommendations.’* 
Compared  to  previous  studies  this  was  the  most  comprehensive 
amd  indepth  study  of  the  problem.  It  covered  the  full 
spectrum  of  the  DOD,  but  within  the  scope  of  this  thesis, 
the  author  only  addressed  the  procurement  process. 

The  Packard  Ccanmission*  s  study  of  the  procurement 
process  revealed  many  of  the  same  findings  observed  in 
previous  government  studies.’*  In  auialyzing  top  level 
management,  the  commission  observed  that,  "There  is  today  no 
single  senior  official  in  OSD  working  full-time  to  provide 
overall  supervision  of  the  acquisition  system. The 
establishment  of  a  deputy  director  in  research  and 
engineering,  and  test  auid  evaluation  by  former  SBCDBFs  Laird 
and  Schlesinger  were  not  sufficient  to  resolve  the  overall 
problem.  The  services  took  on  the  responsibility  themselves 
to  exercise  authority  of  DOD  policy  within  their  branch. 
Unfortunately,  each  service  acted  in  their  own  interests  and 
in  the  long  run  uniformity  in  following  procurement  policy 
deteriorated  due  to  lack  of  centralized  control. 

Twelve  years  after  SECDEF  Schlesinger  reported  to 
Congress  and  promoted  the  use  of  prototyping,  the  Packard 
Commission  re-emphasized  the  need  for  prototyping.  The 
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commission  observed  that  prototyping  did  not  gain  enough 
support  in  the  past  and  that  it  should  receive  much  higher 
priority.  By  doing  so,  the  coitanission  believed  that  a 
service  could  satisfactorily  conduct  a  proof  of  concept 
demonstration  using  the  latest  technology  available  to 
determine  if  a  new  system  in  fact  actually  worked.  If  it 
did  work,  the  prototype  system  would  provide  a  basis  for 
realistic  cost  estimates  before  entering  full  production.*^ 

In  the  area  of  parts,  ccmnponents,  and  subsystems,  the 
ccanmission  observed  the  reliance  on  excessively  rigid 
military  specifications.**  The  Packard  commission  observed 
that  no  matter  how  efficient  DOD  became,  it  could  not  take 
advantage  of  economies  of  scale  euid  mcuiufacture  peurts  as 
cheaply  as  the  commercial  marketplace.  The  products 
developed  to  military  specifications  for  exclusive  military 
use  generally  cost  substauitially  more  than  commercial 
counterparts.  As  eui  exan^le,  the  commission  determined  that 
the  unit  cost  of  a  militarized  computer  microchip  was  3-10 
times  that  of  its  commercial  counterpart.**  As  a  result  the 
ccmimission  recommended  that  only  when  readily  available 
items  did  not  meet  military  requirements  should  new  or 
custom-made  items  be  considered. 

The  commission  did  not  disregard  the  fact  that 
military  computers  needed  to  be  rugged  in  order  to  withstand 
the  rigors  in  the  operational  environment;  but  it  also  noted 
that  industry  standards  for  producing  microchips  had  been 
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greatly  improved  over  the  years.** 

President  Reagan  approved  the  Packard  Commission's 

recommendations  and  signed  National  Security  Decision 

Directive  219  on  1  April  1986  placing  them  into  effect.*® 

DOD  followed  suit  and  revised  its  DOD  directives  and 

instructions  accordingly.  The  Navy  implemented  the  Packard 

commission's  recommendation  for  promoting  the  use  of  NDI  in 

Secretary  of  the  Navy  Instruction  (SBCNAVINST)  4210.7, 

specifically  stating  that: 

It  is  Secretary  of  the  Navy  policy  to 
institutionalize  NDI  consideration  during  the 
acquisition  process  to  eui  extent  that  its  use 
becomes  the  rule  rather  than  the  exception.** 

Defense  Science  Board  (1986) 

The  moxnentum  for  improving  DOD's  procurement  policy 
continued,  and  in  Septerriber  1986  the  Under  Secretary  of 
Defense  for  Research  and  Engineering  Donald  Hicks  tasked  the 
DSB  to  review  progress  regarding  the  recommendations  made 
since  the  last  DSB  study  conducted  in  1978.*^  The  DSB 
consisted  primarily  of  the  same  board  members,  and  they 
reported  that  significant  progress  had  been  made  in  the  past 
eight  years.  The  key  to  the  success  of  bringing  C3I  systems 
into  the  realm  of  current  technology  was  the  CINC's 
involvement.  Since  1982,  DOD  required  each  CINC  to  s\ibmit 
an  annual  Command  and  Control  Master  Plan  containing 
requirements  and  future  plcuis.**  By  involving  the  CINC's  in 
the  planning,  it  gave  planners  a  better  framework  for 
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developing  the  next  generation  of  C3I  systems. 

As  the  years  passed,  remaining  DOO  directives  were 
revised  to  reflect  President  Reagan's  signing  of  National 
Security  Decision  Directive  219.  Specifically,  DOD 
Directive  4120.20,  Development  and  Use  of  Non -Government 
Standards  was  revised  in  March  1988  which  directed 
acquisition  personnel  to  use  nongovernment  steuidards  when 
designing  an  item  \inless  otherwise  specified  by  law, 
multinational  treaty,  or  its  use  was  not  econcanically 
feasible.^* 

As  eui  example,  the  next  sttbproblem  examined  how  the 
Navy  adapted  to  the  chcuiges  in  DOD  procurement  policy  during 
the  period  from  1970  to  1990.  Additionally,  the  next 
s\ibproblem  illustrated  two  C3I  systems  procured  under 
different  policies. 


Subproblem  Number  2 

The  second  subproblem  was  to  determine  the 
operational  effectiveness  and  suitability  of  COTS  computer 
equipment  in  meeting  the  naval  commander ' s  operational 
requirements  in  a  tactical  environment.  To  answer  the 
second  sxibproblem,  I  compared  and  analyzed  two  C2  systems, 
the  Tactical  Flag  Command  Center  Increment  II  and  Joint 
Operational  Tactical  System  I. 

The  second  hypothesis  was  that  COTS  con^)uter 
equii  lent  employed  in  a  tactical  environment  to  support  the 
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Naval  Commander  meets  his  operational  requirements  but  does 
not  meet  the  military  standards  promulgated  under  the  new 
procurement  policy. 

To  familiarize  the  reader  with  the  two  systems  that 
this  study  compared  and  analyzed,  I  have  provided  a  brief 
mission  and  system  description  of  each. 

Baclccrround 

History 

Tactical  Flag  Command  Center  (TPCC) .  The  TFCC 
program  began  in  1971  as  a  research  and  development  program 
for  concept  development  cuid  validation  of  an  automated 
processing  system  for  intelligence  information.  The  program 
began  as  Multi-Source  Processing  System  (MSPS)  and  assvimed 
different  project  names  over  the  course  of  its  development 
(Table  I) .  COMOPTEVFOR  conducted  operational  testing  and 
evaluation  of  MSPS  in  1974  and  validated  the  concept. 
CCS40PTEVFOR  also  concluded  that  the  system  had  the  potential 
for  supporting  the  tactical  commemder  at  sea  auid  recommended 
further  development  be  conducted.  The  Navy  approved  the 
recommendation  for  further  development  in  December  1974.*® 

In  early  1975,  COMOPTEVFOR  conducted  additional  OT&E 
of  MSPS,  then  called  OUTLAW  SHARK  and  identified  areas 
requiring  refinement  to  improve  the  system.  Again, 
COMOPTEVFOR  concluded  that  the  system  had  significant 
potential  for  supporting  the  commander  afloat  with  processed 
intelligence  information.  In  Jiine  1975  the  Navy  esteJalished 
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the  OUTLAW  SHARK  program  as  a  formal  C2  program  and  renamed 
it  TFCC.  From  June  1976  to  July  1977,  COMOPTBVFOR  conducted 
additional  OT&E  of  TFCC,  and  they  observed  deficiencies  that 
were  should  have  been  identified  during  developmental 
testing.®^  For  example,  the  developing  agency  did  not 
verify  system  hardware  and  software  readiness,  stcuidard 
operating  procedures  cuid  manning  requirements . 

In  the  Fall  of  1977  the  Chief  of  Naval  Operations 
(CNO)  suspended  further  development,  pending  review  cmd 
restructuring  of  the  Navy  Command  auid  Control  System 
program.  Over  the  next  two  years,  program  managers  finally 
developed  a  firm  plan  that  received  CNO  approval,  and  the 
Navy  reactivated  the  TFCC  program  in  August  1979.®* 

The  TFCC  program  was  restructured  and  divided  into 
three  incremental  phases.  TFCC  Increment  I  was  a  space 
upgrade  in  aircraft  carriers  (CV/CVNs) .  This  phase  provided 
the  embarked  tactical  commcuider  with  existing  Navy  equipment 
that  included  vertical  status  boards.  Naval  Tactical  Data 
System  (NTDS)  consoles,  and  communications  capabilities  to 
support  his  decision  making.  TFCC  Increment  II  incorporated 
the  Flag  Data  Display  System  (FDDS) ,  an  updated  version  of 
OUTLAW  SHARK.  FDDS  performed  automated  information 
processing  and  display  and  also  provided  for  the 
manipulation  and  display  of  tactical  information.  The  Navy 
installed  the  system  onboard  six  aircraft  carriers.  TFCC 
Increment  III  provided  for  evolutionary  improvements  which 
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were  not  yet  defined. 

Over  the  next  several  years  the  Navy  conducted 
additional  phases  of  DT  and  OT  (Figure  l)  that  refined 
system  performance  prior  to  OPEVAL. 

Joint  Operational  Tactical  System  I  (JOTS  I) .  JOTS  I 
began  as  a  fleet  initiative  under  the  sponsorship  of  the 
Commander  in  Chief,  Atlantic  Fleet  ( CINCLANTFLT) ,  funded 
under  a  rapid  prototype  program  that  was  developed  to 
satisfy  Operational  Requirement  249-094-89  in  August  1989.** 
The  requirement  was  for  a  downsized  desktop  computer-based 
tactical  system  to  provide  information  transfer,  display  and 
decision  aids  for  afloat  users.  The  fleet  CINCs  used  JOTS  I 
to  address  deficiencies  in  processing  wide  area  surveillance 
infoimnation,  and  to  provide  tactical  decision  aids  (TDAs) 
not  being  met  by  other  afloat  C2  systems. 

Of  the  eight  remaining  aircraft  carriers  that  did  not 
have  FODS  installed,  JOTS  I  served  as  the  interim  C2  system 
In  TFCC  for  the  embarked  tactical  commcuider. 

Mission  Description 

TFCC  Increment  II.  TFCC  Increment  II  or  FDDS  were 
synonymous  terms.  This  study  treated  these  as  one  and  used 
the  acronym  FDDS.  The  FDDS  was  an  automated  data  processing 
system  designed  to  process  and  correlate  surveillance 
information  from  external  sources  received  via  various  modes 
of  communications  (i.e.,  satellite,  radio)  as  well  as 
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organic  sources  within  the  carrier  battle  group.  The  system 
performed  data  base  management  functions  and  contact 
correlation,  and  it  displayed  the  alphanumeric  and  graphic 
information  on  a  large  screen  display  (LSD)  for  use  by  the 
embarked  tactical  commander.  As  a  final  note,  the  system 
was  not  designed  to  counter  any  specific  threat  as  stated  in 
its  operational  requirement.” 

JOTS  I.  The  JOTS  I  system  processed  surveillance 
information,  treuisferred  tactical  information  to  other  C2 
systems,  such  as  Tomahawk  Weapons  Control  System  (TWCS) ,  and 
provided  TDA's  in  support  of  the  tactical  commander.  JOTS  I 
provided  the  capability  to  receive  data  via  various  modes  of 
communications  (i.e.,  satellite,  radio)  for  processing;  to 
interface  directly  with  other  C2  systems  (i.e.,  FDDS,  TWCS, 
Prototype  Ocean  Surveillance  Terminal  (POST) ) ;  and  to 
perform  contact  association/matching  and  data  base 
management.  JOTS  I  also  generated  and  exchanged  color 
tactical  graphic  displays;  and  provided  Point  of  Intended 
Movement  capability,  satellite  vulnerability  capability,  and 
TDA's  for  supporting  subordinate  warfare  commanders.  JOTS  I 
was  not  designed  to  counter  any  specific  threat  as  stated  in 
its  operational  requirement.” 

System  Description 

FDDS ■  The  FDDS  (AN/USQ- 81 (V) )  system  consisted  of 
the  following:” 

1.  AN/UYK-19  con^juter  (64k  bites  of  memory) 
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2.  Associated  peripherals 

3.  Operator  Interface  Terminals  (OITs)  networked  in 
TPCC,  Carrier  Intelligence  Center  (CVIC) ,  Combat  Direction 
Center  (CDC) ,  and  Supplementary  Plot  (SUPPLOT) 

4.  Disk  memory  (2Mb) 

5.  Magnetic  Tape  Recorder 

6.  Hard  copy  Plotter 

7 .  Sanitization  Terminal 

JOTS  I.  The  JOTS  I  system  consisted  of  the 
following:** 

1.  Five  HP-9020A/C  desktop  con^uters,  each  with  a 
built-in  monitor  and  keyboard 

2.  Rocky  Mountain  BASIC  operating  system  written  in 
BASIC  language 

3.  Hard  drive  (either  55Mb  or  l20Mb) 

4.  Typically  configured  as  a  five  terminal  system 
connected  via  a  local  area  network  in  TPCC,  CVIC,  CDC,  and 
SUPPLOT.  The  JOTS  master  terminal  was  located  in  the  TPCC 
and  functioned  as  the  network  server. 

5 .  Hardcopy  printer /plotter 

6.  The  system  also  Included  peripherals  such  as  a 
large  screen  display,  remote  monitors  and  remote  control 
devices  (digipads)  for  manipulating  the  displays. 
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TFCC  Acquisition  Cycle 


The  TFCC  program  as  depicted  in  Table  I  and  Figure  l 
and  as  described  in  the  historical  background  section  has 
had  a  long  acquisition  cycle  that  from  the  time  of  its 
inception  spanned  almost  17  years.  In  the  program's  earlier 
years,  COMOPTEVFOR  observed  that  TFCC  lacked  direction  when 
it  conducted  its  operational  assessments.  It  was  a  program 
that  was  a  first  of  its  kind,  and  it  experienced  growing 
pains,  most  notably  in  1977  when  CNO  suspended  it. 

In  looking  at  the  progression  of  the  TFCC  program, 
TFCC  was  a  good  example  of  what  the  DSB  had  stated  edsout  the 
xiniqueness  of  C3I  systems  acquisition.  It  was  complex  and 
evolutionary  in  nature  as  well  as  software  intensive. 

When  the  TFCC  program  regained  its  stature  in  mid- 
1979,  the  Navy  approved  two  engineering  development  models 
(EDM's)  for  production.  The  first  system  reached  initial 
operating  capability  (IOC)  in  mid- 1983  and  was  installed 
aboard  the  aircraft  carrier  USS  AMERICA.”  The  program  was 
now  on  track,  cuid  it  adhered  closely  to  the  DOD  Directive 
5000.3  Test  and  Evaluation  for  conducting  OT  early  on 
following  the  completion  of  DT  (Figure  l  and  Table  I) .  with 
the  exception  of  one  test  phase  in  September  1981,  both  OT 
and  DT  were  combined  due  to  limited  availability  of 
resources . 

Figure  1  also  depicted  the  associated  costs 
acctimulated  over  the  program's  lifecycle.  Beginning  with 
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program  reactivation  in  1979,  the  initial  R&o  costs  were 
over  $50  million  and  continued  to  increase  incrementally 
over  the  years,  taUcing  into  account  inflation.  By  the  end 
of  FY89,  the  total  RDT&E  costs  had  reached  $149,043,000; 
procurement  costs  for  six  FDOS  systems,  $102,839,000;  ship's 
new  construction,  $74,741,000;  euid  operations  and 
maintenance,  $56,876,000.*** 

JOTS  Acquisition  Cycle 

The  JOTS  acquisition  cycle  was  unlike  any  C3I  program 
under  procurement.  The  system  received  its  beginnings  in 
1982  under  the  direction  of  then  RADM  Tuttle,  Comnander 
Carrier  Group  Two.  JOTS  was  a  low  cost  system  that  provided 
Navy  management  functions  hosted  on  a  comnercial  HP9020 
desktop  computer.  The  costs  were  so  slight  that  it  was 
supported  solely  by  Operational  and  Maintenance  Navy  funds, 
funds  that  paid  for  fuel  cUid  parts.  The  capcd>ility  of  the 
JOTS  system  grew  in  the  ensuing  years  and  gained  widespread 
support  from  the  fleet.  For  a  small  investment  of 
approximately  $55,000  in  ship's  OPTAR,  commanding  officers 
had  the  capcdsility  to  fill  a  void  in  their  C2  and  also 
conduct  operational  planning. 

In  essence,  the  fleet  supported  the  program  as 
opposed  to  a  sponsor  in  Washington,  DC.  The  proliferation 
of  JOTS  systems  was  not  a  problem,  xintil  maintenance  was 
required.  Unfortunately,  logistic  support  for  the  system 
did  not  exist  within  the  Navy  supply  system  and  the  vendor 
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provided  maintenauice . 

Eventually,  the  JOTS  program  was  given  a  sponsor  who 
would  oversee  the  program  aind  who  had  the  authority  to 
develop  a  support  structure  to  include  necessary 
docximentation  and  logistic  support.  Figure  2  depicted  the 
associated  costs  acciimulated  over  the  program's  lifecycle. 
Because  JOTS  was  a  fleet  initiative,  R&D  costs  were  zero  for 
the  first  7  years  and  only  began  to  accumulate  slightly  when 
a  program  sponsor  was  identified  and  took  charge  in  FY  89 . 
Procurement  costs  for  219  JOTS  systems  acquired  by  fleet 
units  were  approximately  $2,200,000;  and  operations  euid 
maintenance  costs,  $500,000  (Figure  2). 

QT&B  Process 

OT&E  adhered  closely  to  DOD  policy  auid  followed  the 
guidance  of  conducting  OT  30  days  after  the  completion  of 
DT.  FDDS  and  JOTS  I  operational  assessments  were  conducted 
follof^ing  the  completion  of  DT  (Table  I  and  II)  .  Hillman 
Dickinson,  Director  of  C3  systems,  JCS,  discussed  the 
difficulty  in  performing  realistic  evaluation  of  C3  systems, 
but  lauded  the  Navy's  efforts  in  this  area.” 

In  determining  whether  sufficient  testing  was 
conducted  prior  to  OPEYAL,  it  did  not  appear  to  have  been 
the  case  for  FDDS  or  JOTS  I.  If  sufficient  DT  and  OT  were 
conducted  to  resolve  deficiencies  and  make  corrections,  the 
results  of  OPEVAL  (Table  IV)  would  have  indicated  higher 
satisfactory  grades  thcui  those  actually  acquired. 
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The  FDDS  euld  JOTS  test  environments  were 


operationally  realistic,  but  limited  in  scope  in  that  they 
did  not  duplicate  operations  in  a  worst  case  scenario.  The 
operational  testing  for  FDDS  was  conducted  in  a  non*ccxnbat 
environment  in  the  North  Arabiam  Sea  for  a  period  of  14  days 
aboard  USS  AMERICA.  (CV  66)  and  supported  the  carrier  group 
comnamder,  his  staff,  and  USS  AMERICA'S  operations 
department.  Testing  was  of  sufficient  length  in  time  to 
evaluate  all  but  one  test  objective,  maintainability. 

The  operational  testing  for  JOTS  was  performed  in 
conjunction  with  a  fleet  training  exercise  off  the  Atlamtlc 
coast  under  simulated  wartime  conditions.  JOTS  was 
installed  aboard  USS  FORRBSTAL  (CV  59)  amd  supported  the 
Carrier  Group  SIX  commander.”  As  discussed  in  subproblem 
niunber  one  regarding  the  validity  of  the  operational  testing 
and  evaluation  of  systems  under  procurement,  I  observed  that 
both  FDDS  and  JOTS  both  adhered  to  the  guidelines  amd 
procedures  for  testing  in  DOD  Directive  5000.3,  Test  and 
Bvaluation.” 

The  test  environment  did  not  Include  the  effects  of 
EMP  on  the  system.  However,  other  sources  of  information 
that  discuss  the  effects  of  EMP  in  a  nuclear  war  were 
classified  and  beyond  the  scope  of  this  thesis. 

Operational  evaluation  (OPEVAL) ,  the  phase  of 


operational  testing  that  provides  results  and  conclusions 
that  determine  if  a  system  vinder  procurement  will  go  into 
full  production  was  used  as  the  basis  for  conducting  the 
comparison  emd  analysis  of  FDDS  and  JOTS.  Prior  to  OPEVAL, 
a  system  will  have  all  deficiencies  identified  in  the 
leJsoratory  environment  corrected  or  resolved  and  receive 
certification  from  the  program  manager  that  the  system  is 
ready  for  OPEVAL. 

Opegfl-tional  effectiveneae. 

The  principle  test  objective  was  to  evaluate  the 
overall  effectiveness  of  each  system  to  provide  timely  and 
accurate  data  to  support  the  afloat  comrieuider  euid  to 
determine  which  one  was  better.  Details  in  system 
performance  were  subdivided  and  evaluated  by  function  amd 
summarized  in  Table  III.  The  specific  functions  were  graded 
SAT,  UNSAT,  and  UNRESOLVED  based  on  COMOPTEVFOR ' S 
observations  and  analysis.  Functions  that  were  unique  to 
one  system  which  did  not  lend  themselves  to  comparison  were 
evaluated  for  their  value-added  potential/capability  to 
support  the  tactical  commander.  In  areas  where  functions 
were  the  same  or  similar  but  identified  by  different 
terminology,  annotations  were  made  in  TeQjle  III  notes  for 
clarification. 

COMOPTEVFOR  observed  that  FDDS  and  JOTS  did  not 
adequately  demonstrate  the  capability  to  support  the  afloat 
commander  overall.*^  FDDS  received  UNSAT  grades  in  all 
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areas  of  operational  effectiveness  with  the  exception  of 
two.  Those  two  areas  were  part  of  JOTS  evaluation  (security 
and  TDA's)  and  not  applicable  to  PDDS.  TPCC,  CDC,  CVIC,  and 
SUPPLOT  were  certified  as  secure  spaces  in  which  to  house 
classified  materials  amd  equipment  during  aircraft  carrier 
construction  auid  therefore  not  identified  as  an  issue  in  the 
TEMP.  TDA's  were  not  yet  developed  into  the  FDOS  software 
auid  therefore  were  not  evaluated.  JOTS  I  received  5  of  7 
UNSAT  marks  in  operational  effectiveness.  Operational 
effectiveness  issues  were  evaluated  by  how  effectively 
information  was  hamdled  by  function.  Details  were  provided 
in  the  following  paragraphs. 

One  of  the  primary  functions  of  a  C2  system  was  to 
determine  how  effectively  information  was  processed  amd 
excheuiged  over  the  communication  circuits.  COMOPTEVFOR 
observed  that  FDDS  and  JOTS  I  were  both  inadequate  in 
performing  this  function.  FDDS  was  not  sufficiently 
automated  to  handle  the  volume  of  duplicate  messages 
received  over  the  multiple  communication  circuits. 

(Multiple  communication  circuits  provided  a  redundant  as 
well  as  alternate  means  of  receiving  messages  in  the  event 
of  communications  problems . )  COMOPTEVFOR  observed  that 
information  was  backlogged  for  processing  and  required 
extensive  operator  intervention  to  alleviate  the  buildup. 

In  the  instances  of  duplicate  messages,  FDDS  reprocessed  the 
information  several  times  as  opposed  to  recognizing  the 
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duplicate  and  discarding  it.‘® 

JOTS  I  possessed  the  capacity  to  store  a  greater 
volume  of  incoming  messages  in  buffers  but  failed  to 
adequately  process  all  messages.**  The  percentage  of 
unprocessed  messages  was  small  but  statistically  significant 
enough  to  mcdce  a  difference  in  the  accuracy  of  the  tactical 
picture  (the  classification  of  the  data  precluded 
disclosure) .  The  most  importamt  messages  that  were  not 
processed  were  contact  reports  (positional  reports  of 
friendly,  enemy,  or  neutral  surface,  subsurface,  or  air 
contacts)  that  contributed  to  the  development  of  an  accurate 
tactical  picture.  The  inadequacy  of  this  JOTS  I  fxanction 
led  to  a  degraded  tactical  picture  and  affected  the  Force 
Over- the -Horizon  Track  Coordinator’s  and  Anti- surface 
Warfare  Commcuider's  abilities  to  effectively  fulfill  their 
mission  requirements.*’ 

To  compound  the  information  processing  problem,  FDDS 
euid  JOTS  I  did  not  adequately  correlate  the  incoming  contact 
reports  correctly.  The  correlation  and  matching  was 
primarily  an  FDDS  euid  JOTS  I  function;  however,  operator 
intervention  was  required  in  many  cases.  The  correlation 
process  involved  the  use  of  contact  attributes  (i.e.,  unique 
information,  such  as  ship  name,  hull  number,  international 
call  sign,  ship  class,  geographic  position)  in  performing 
the  calculation.  In  the  FDDS  system,  a  scoring  system  was 
used  by  assigning  values  (O-lOO)  to  attributes.  The 
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numerical  value  assigned  was  based  on  the  uniqueness  of  the 
attribute.  Unfortunately,  the  algorithm  was  sinplistic  and 
operator  intervention  was  required  in  resolving  52  percent 
of  the  ambiguous  reports.** 

JOTS  I  experienced  a  similar  problem.  Its 
correlation  algorithm  was  even  more  simplistic  than  FDDS. 

All  attributes  were  not  weighted;  and  instead,  JOTS  I 
coR^ared  the  attribute  information  between  contacts  and 
conducted  a  geo- feasibility  analysis  to  determine  the 
likelihood  that  a  new  contact  was  in  fact  an  updated  one. 
Geo- feasibility  analysis  was  a  calculation  based  on  time, 
distance  and  position  of  two  possible  candidates  within 
proximity  of  one  another  as  being  the  'me  one.  The 
calculation  determined  the  likelihood  that  a  contact  (air, 
surface,  or  sxibsurface)  could  be  the  same  based  upon  the 
difference  in  time,  disteuice  and  speed. 

Ctece  incoming  message  reports  were  processed 
(correlated  if  they  happen  to  have  been  contact  reports) , 
the  data  was  stored  in  various  data  bases  depending  on  the 
type  of  message.  The  FDOS  system  was  inadequate  in  data 
base  management.  The  procedures  for  the  manual  input  and 
meuiipulation  of  data  were  time-consxjming  and  labor  intensive 
without  regard  to  man-machine  interface  considerations. 

JOTS  I  performed  data  base  management  funccx'.ns 
satisfactory.  The  hardware  capacity  ened>led  the  system  to 
store  additional  types  of  data  that  included  oceanographic, 
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weather,  overlay,  and  satellite  vulneredsillty  data.  No 
deficiencies  were  observed  in  the  autcsnatic  posting  and 
updating  of  such  information.** 

A  key  element  in  supporting  the  tactical  commander's 
decision  making  process  was  to  evaluate  how  effectively  data 
was  assimilated  and  con^rehendible  by  meauis  of  tactical 
displays.  Once  Information  was  stored,  presenting  the  data 
in  an  the  most  effective  manner  to  encdale  the  tactical 
ccmnander  to  quickly  assimilate  it  cuid  maJce  a  decision.  The 
most  effective  means  was  through  visual  display.  By  having 
a  clear,  concise  graphic  display  of  how  enemy,  friendly,  euid 
neutral  forces  were  arrayed,  the  tactical  commcuider  could 
then  make  an  informed  decision  given  the  intelligence 
available . 

FDDS  failed  to  display  a  tactical  picture 
effectively.  The  alphanumeric  data  cluttered  the  graphic 
displays  and  degraded  the  tactical  picture,  nteUcing  the 
display  xinreadable .  Additionally,  the  tactical  displays 
were  static  and  presented  only  a  "snapshot”  view  of  the 
tactical  situation.  JOTS  I  provided  a  color  display  of 
friendly,  enemy,  eUid  neutral  contacts  in  near- real  time  and 
COMOPTEVPOR  observed  that  no  deficiencies  were  observed.’^ 
Unique  to  JOTS  was  the  availability  of  TDA's  to 
enhance  the  tactical  commander's  decision-making.  TDA's 
varied  from  simple  range  euid  bearing  functions  to  more 
complex  calculations,  such  as  determining  satellite 
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vulnerability  periods  (time  frames  in  which  military  forces 
were  subject  to  exploitation  by  non-US  satellites) .  These 
aids  were  automated  functions  that  gave  him  a  significant 
advantage  over  manual  methods  of  calculation  and  were 
readily  available  on  the  con^uter.  Also,  TDA's  gave 
commanders  the  flexibility  to  perform  "what  if"  calculations 
quickly  amd  then  modify  operational  plans  as  necessary. 

Security  was  not  a  test  objective  during  FDDS  testing 
and  was  not  evaluated.  However,  security  was  a  test 
objective  in  JOTS  testing,  and  testing  revealed  that 
adequate  security  measures  did  not  exist  to  preclude 
unauthorized  personnel  or  JOTS  operators  from  accessing  amd 
altering  the  source  code  of  the  software.  The  impact  would 
have  been  severely  disrupted  operations.’* 

Operational  suitability 

Operational  suitability  was  described  as  the  degree 
in  which  a  system  could  be  placed  satisfactorily  in  the 
field  with  consideration  given  to  factors  such  as 
operational  availability,  maintainad^ility,  reliability, 
logistic  supportability,  survivedjility,  training, 
interoperability,  safety,  human  factors,  emd 
compatibility.’*  FDDS  euid  JOTS  I  were  tested  in  all  areas 
of  suitodaility  except  survivability  in  a  nuclear 
environment . 

Both  FDDS  and  JOTS  demonstrated  satisfactory 
performance  in  system  relicQaility  (Tables  III) .  During  the 
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OPEVAL,  PDDS  demonstrated  an  MTBF  of  322  hours  (criterion  72 
hours),  exceeding  the  criterion  by  4.47  times.  Although 
FDDS  operated  for  only  322  hours  (criterion  720)  without  a 
critical  failure  in  hardware,  COMOPTEVFOR  observers 
subjectively  evaluated  the  system's  ffTBMCF  as  satisfactory. 
As  for  MTBMCF  (SW) ,  FDDS  halted  seven  times  in  322  hours 
(criterion  4  hours),  exceeding  the  criterion  by  11.5  times. 
JOTS  operated  at  least  876  hours  without  any  hardware 
failures  (criterion  500  hours  and  250  hours  for  MTBMCF  (HW) 
and  MTBF  (HW) ,  respectively) .  Eight  software  faults  were 
observed  but  JOTS  I  still  exceeded  the  criterion  by  7.26 
times. 

Maintainability  for  FDDS  was  unresolved  because  one 
repair  job  did  not  represent  a  statistically  significant 
san^le  size  for  MTTR.  JOTS  received  a  satisfactory  grade. 
Not  discussed  in  the  OPEVAL  report  was  the  importance  of 
availability  in  spare  parts  which  determined  how  long  it 
took  to  repair  a  discrepancy.  This  variable  was  directly 
relatec  to  logistic  support ability  of  the  system  and  given 
the  lack  of  spare  parts,  the  operational  effectiveness  of 
the  system  could  severely  be  degraded.  Further  details  were 
discussed  under  logistic  supportability .  The  mean  time  to 
restore -operator  (MTTR-0)  was  22.6  minutes  (criterion  4 
min) .  FDDS  failed  to  achieve  this  threshold  by  a  factor  of 
5.6  times. 

The  MTTR  for  JOTS  I  was  not  demonstrated  because  no 
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mission  critical  failures  were  observed.  COMOPTBVFOR 
subjectively  evaluated  this  objective  as  satisfactory,  based 
on  its  observations.  For  MTTR-O,  JOTS  demonstrated  an  MTTR- 
0  of  15  minutes,  exceeding  the  criterion  by  a  factor  of  four 
times. 

In  coii^aring  the  availability  of  both  systems,  FDDS 
received  a  performance  evaluation  of  UNSAT  whereas  JOTS 
received  a  SAT  (Tables  IV) .  FDDS  demonstrated  an  A(o)  of 
.96,  based  on  310.2  hours  of  uptime  out  of  322  hours. 

JOTS  demonstrated  an  A(o)  of  .99,  based  on  874  hours  of 
uptime  out  of  876  hours.  The  two  hours  of  downtime  were  the 
result  of  software  faults  attributed  to  information 
overload.’* 

In  the  area  of  logistic  supportability,  FDDS  was 
evaluated  as  SAT  and  JOTS,  UNSAT  [Table  IV)  .  The  FDDS 
OPBVAL  report  stated,  "No  deficiencies  were  noted  in  spare 
parts  required  to  support  TFCC/FDDS  and  associated 
hardware."’*  Given  the  long  procurement  time  of  the  system, 
the  Navy  supply  system  was  able  to  establish  an  adequate 
integrated  logistic  support  structure  to  provide  for  the 
system. 

On  the  other  hcuid,  JOTS  did  not  have  an  integrated 
logistic  support  program.”  A  maintenance  program  was  not 
yet  in  existence,  and  spare  parts  and  consumable  items  were 
not  adequately  stocked  in  the  Navy  supply  system.  Instead, 
logistic  support  was  provided  by  the  contractor.  In  Buying 
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NDI .  one  of  the  problems  discussed  in  using  NDI  (i.e.,  COTS) 
was  that  the  lead  times  for  estaiblishing  an  infrastructure 
to  provide  the  necessary  logistic  support  could  exceed  the 
time  expended  in  fielding  the  system.’*  This  was  certainly 
true  of  JOTS.  Additionally,  the  risks  must  have  been 
acceptcdsle,  or  the  program  manager  would  not  have  onsidered 
it  as  eui  option. 

In  the  area  of  training,  both  FDDS  and  JOTS  programs 
were  inadequate  in  providing  the  necessary  training  to  users 
in  operating  the  systems  with  proficiency.  The  FDDS 
training  program  did  not  fully  cover  proper  data  base 
management,  trouble-shooting  procedures  for  the  system,  and 
team  training  which  integrated  officers  and  enlisted 
personnel.”  For  JOTS  I,  a  formal  Chief  of  Naval  Education 
Training  program  had  not  yet  been  established.  Training  was 
ad  hoc  and  provided  by  fleet  mobile  training  teams  and 
contractors . 

Con^jatibility  of  the  systems  operating  within  the 
established  environment  differed  between  FDDS  and  JOTS  I . 

The  FDDS  system  was  con5)atible  with  the  physical, 
electrical,  and  electronic  operating  environment.*^  JOTS  I 
was  not  physically  compatible  within  the  operating 
environment.  Thw  system  was  installed  on  a  space  available 
basis  and  interfered  in  the  maintenance  of  adjacent 
equipment  within  the  same  compartment.** 

In  comparing  the  technical  docxamentation  for 
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maintaining  PDDS  and  JOTS,  both  were  inadequate  in  providing 
logical  and  con^lete  troubleshooting  procedures,  cuid  errors 
were  noted  in  performing  system  operations.” 

In  the  area  of  survivability,  FDDS  and  JOTS  received 
evaluations  of  SAT,  although  testing  was  conducted  only  to  a 
limited  degree.  COMOPTEVFOR  observed  that  TPCC/FDDS  was  no 
more  or  less  vulnerable  to  battle  damage  than  other  systems 
operating  in  the  shipboard  environment.  However,  that 
conclusion  was  made  with  the  caveat  that  testing  was 
conducted  in  a  non* combat  environment,  euid  the  system  was 
not  subjected  to  BMP  or  ECM.”  JOTS  I  testing  revealed  no 
deficiencies  either.  However,  in  this  case,  testing  the 
system  for  EMP  effects  was  not  an  operational  requirement  of 
the  system.” 

Although  no  actual  testing  was  performed  to  evaluate 
JOTS  survivability  against  a  conventional  threat,  the  single 
JOTS  systems  installed  aboeurd  the  USS  STARK  and  USS  ROBERTS 
proved  very  survivable  against  the  shock  and  vibration  from 
a  single  air- to- surface  missile  and  mine  attack  in  the 
Persian  Gulf.”  While  on  patrol  in  the  Persian  Gulf,  the 
USS  ROBERTS  Struck  eui  Ireuiian  mine  which  knocked  out  much  of 
her  electronics  with  the  exception  of  its  Hewlett-Packard 
computer.  In  a  separate  incident,  the  USS  STARK  while  on 
patrol  in  the  Persian  Gulf  was  struck  by  an  Exocet  missile 
that  cracked  her  Combat  Information  Center  displays,  putting 
her  combat  systems  out  of  operation.  Within  that  same 
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compartment,  the  Hewlett-Packard  computer  was  found  to 
remain  operational.*'' 

In^ortant  to  system  survivability  were  the  effects  of 
BMP.  COMOPTEVFOR  Stated  that  BMP  testing  was  a  limitation 
to  scope  in  its  testing,  and  therefore  was  not  evaluated. 
Nonetheless,  the  effects  of  BMP  must  be  a  considered.  The 
information  regarding  this  siibject  was  beyond  the 
classification  of  this  thesis,  but  is  avall2d)le  in  other 
sources . 

Discussed  to  a  limited  degree  in  the  FDDS  OPBVAL 
report  was  system  survivability  when  operating  in  a  degraded 
mode.  COMOPTBVFOR  tested  FDDS  imder  degraded  operations  by 
denying  the  tactical  comm^mder  access  to  incoming 
information  from  ashore  facilities  and  Intracommunications 
within  the  battle  group  and  simulated  loss  of  two  terminals. 
Voice  circuits  and  other  communication  modes  provided  the 
means  of  keeping  the  tactical  commander  cUbreast  of  the 
situation.  The  additional  results  of  this  simulated  testing 
were  classified,  but  the  information  may  be  found  in  the 
FDDS  OPBVAL  report.** 

System  interopercdsillty  was  evaluated  by  assessing 
how  well  FDDS  and  JOTS  I  operated  with  other  Navy  systems 
that  Included  shore-based  facilities  and  other  shipboard 
systems.  COMOPTEVFOR  observed  that  FDDS  demonstrated 
satisfactory  performance  in  operating  with  both  Navy  shore 
and  shipboard  systems  without  problems.**  JOTS  I  did  not 
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interface  with  the  Officer  in  Tactical  Comnand  Information 
Exchange  System  (OTCIXS)  communications  net  adequately.*® 
When  problems  were  observed  with  OTCIXS  coiranunications,  it 
caused  JOTS  I  to  generate  software  faults  intermittently. 

The  cause  for  the  problem  was  un]cnown  and  required  further 
investigation  in  COMOPTEVFOR' s  recommendations.’^ 

Operational  suitability  also  included  a  personnel 
side  which  considered  the  hinnan  factors  aspect. 

COMOPTEVFOR '  s  evaluation  of  htaman  factors  deficiencies  was 
based  upon  subjective  observations,  euid  both  systems  were 
evaluated  as  unsatisfactory.  The  FDDS  system  was  installed 
such  that  the  printer  was  not  located  in  the  immediate 
vicinity  of  the  operator  for  retrieval  of  paper  products. 
Additionally,  the  installation  did  not  allow  easy  access  by 
personnel  performing  preventative  and  unscheduled 
maintenance . 

The  human  factors  problem  was  different  in  JOTS  I  and 
focused  on  the  mcui-machine  interface  aspect  related  to  the 
software.  The  JOTS  I  functions  were  primarily  menu  driven 
by  layers  upon  layers  of  submenus  in  different  partitions  of 
the  system.  Determining  the  status  as  well  as  operating  in 
any  partition  required  extensive  knowledge  of  getting  around 
the  software  to  access  information  or  troubleshoot  software 
problems.” 

COMOPTEVFOR  evaluated  the  safety  features  of  both 
FDDS  and  JOTS  I  as  unsatisfactory.  FDDS  was  safe  for 
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operator  use  but  unsafe  to  maintenance  personnel .  The 
latches  housing  the  computer  disk  were  designed  such  that 
maintenance  personnel  could  injure  themselves  from  the 
frame's  sharp  metal  edges  in  atten^ting  to  gain  access  to 
the  storage  area.  JOTS  I  was  potentially  a  physical  and 
electrical  hazard  to  personnel  at  or  near  the  terminal. 
Components  were  not  mounted  or  secured  to  racks,  and  cabling 
was  not  contained  within  cable  conduits  nor  placed  under 
deckplates .  ** 
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CHAPTER  FIVE 


CONCLUSIONS  AND  RECOMMENDATIONS 

Conclusions 

The  hypothesis  of  the  first  svibproblem  is  that  the 
new  DOD  procurement  policy  is  better  than  the  old  one  in 
terms  of  streamlining  the  acquisition  cycle  and  associated 
costs.  I  conducted  euialysis  which  included  the  DOD 
management  and  orgauiization  for  procurement;  DOD  directives 
and  instructions;  the  use  of  mil -specs  versus  commercial 
item  descriptions  and  the  threat.  I  reached  the  following 
conclusions . 

The  acquisition  process  was  improved  and  streamlined. 
Recognition  of  the  problem  was  brought  to  light  by 
government  studies  to  analyze  the  DOD's  methods  of  business 
practice.  The  studies  did  in  fact  identify  a  significant 
problem  in  DOD's  procurement  orgeuiization  euid  mcuiagement. 

The  Pitzhugh,  Grace,  and  Packard  studies  conducted  in  the 
1970 's  to  1980 's  identified  the  magnitude  of  the  procurement 
problem.  The  lessons  learned  by  the  DOD  in  the  Fitzhugh 
study  were  relearned  as  evidenced  in  subsequent  Grace  and 
Packard  studies.  In  addition,  the  DSB  studies  conducted  in 
1978  and  1987  brought  to  light  the  specific  problems  of 
command  and  control  systems  procurement. 
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Inclement ing  changes  to  deeply  rooted  procurement 
problems  was  a  slow  and  tedious  process.  Because  the  DOD 
was  a  conglomeration  of  orgemizations  that  worked 
independently  of  one  another,  it  was  that  much  more 
difficult  to  resolve.  In  making  administrative  chcuiges  that 
cut  across  the  board  of  the  DOD  and  by  establishing 
additional  staff  positions  within  OSD  to  oversee  procurement 
two  major  steps  were  taken  to  solve  some  of  the  problems. 

The  government  studies  repeatedly  reached  conclusions  that 
recommended  improving  the  acquisition  management 
organizat  .on.  DOD  made  changes  to  its  directives  and 
instructions  to  provide  program  managers  the  necessary 
guidance  to  do  their  jobs. 

Following  the  administrative  restructuring  and 
reorganization,  tire  DOD  next  re-educated  its  program 
managers.  (Program  managers  needed  to  re-evaluate  their 
decision-making  process  by  breaking  away  from  the  old 
practice  of  buying  new  systems  and  attaching  every 
conceivable  item  with  a  mil -spec.)  The  overuse  of  mil -specs 
cind  mil-stds,  another  deeply  rooted  problem,  plagued  program 
managers.  DOD  directives  included  recommendations  that  the 
tour  length  for  program  managers  be  increased.  Thereby 
increasing  their  overall  corporate  knowledge  in  both  the 
management  and  technical  aspects  of  procurement. 

The  second  hypothesis  was  that  COTS  coir^juter 
equipment  employed  in  a  tactical  environment  to  support  the 
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Naval  Commander  afloat  met  his  operational  requirements  but 
did  not  meet  the  military  standards  promulgated  under  the 
new  procurement  policy. 

JOTS  I  did  not  meet  the  operational  requirement  of 
the  tactical  commauider.  Although  it  surpassed  FDDS  in 
hardware  and  software  capabilities,  JOTS  was  not 
significantly  better  than  FDDS  in  supporting  the  tactical 
commcuider ' s  C3I  operational  requirements.  Tnose 
effectiveness  and  operational  issues  better  suited  to  meet 
the  requirements  of  the  tactical  commander  in  JOTS  were 
offset  by  poor  logistic  supportability,  inadequate 
compatibility  and  interoperability  whe.i  compared  to  the 
FDDS,  JOTS  I  did  provide  TDA's  not  available  in  FDDS  and 
did  prove  to  be  an  invaluable  planning  tool  for  the  tactical 
commander  in  his  decision-making. 

In  fielding  a  rapid  prototype  system  such  as  JOTS  I, 
there  were  some  obvious  tradeoffs  identified  in  using  COTS 
computers  in  technical  performance,  operational 
effectiveness  and  suitability.  In  COMOPTEVFOR ' s 
conclusions,  both  systems  were  evaluated  as  having  the 
potential  to  be  operational  effective  and  suitable  once 
corrections  to  those  deficiencies  were  corrected. 

Recommendations 

This  thesis  covered  an  important  period  in  time  in 
which  the  US  Navy  focused  it  attention  to  a  Soviet  threat. 
Witb  the  collapse  of  the  Soviet  Union,  reshaping  of  the 
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military,  and  the  US  Navy's  role  today,  I  recommend  the 
following  for  further  study. 

The  potential  for  military  involvement  in  regional 
conflicts  is  probably  the  most  likely.  Further  study  of  how 
to  use  C3I  systems  in  a  low  intensity  conflict  environment 
should  be  investigated. 

With  the  advent  emerging  new  technologies  in 
to..ftware,  the  likelihood  of  the  viruses  infecting  software 
will  probcdsly  also  increase.  The  inpact  of  such  an 
occurrence  could  seriously  inpair  C2  operations  at  any 
warfighting  level.  This  potential  threat  warrants  further 
study  in  determining  measures  of  countering  it. 

In  that  same  spirit  of  countering  the  effects  of 
viruses,  further  study  should  be  conducted  to  develop  more 
effective  counter  countermeasures  in  denying  enemy  access  to 
their  C3I  systems  while  protecting  our  own. 

Finally,  Desert  Storm  demonstrated  to  the  military 
that  combining  forces  in  a  joint  effort  provided  a 
synergistic  effect  in  combatting  the  enemy.  As  the  US 
military  places  greater  enphasis  in  joint  oper&wions, 
further  research  should  be  conducted  to  inprove 
interoperability  of  Navy  C3I  systems  in  a  joint  environment. 
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APPENDIX  A 


TABLE  I 

JOTS  CniRONOIiOGY 


Project  Name 

Date (8) 


DT&E  Date(s)  QT&£ 


K310-5  Jul  88  Aug  88 

JOTS  I  lOT&E  Jxxn-Jul  89  Aug  89 


NOTES: 

Data  extracted  from  JOTS  TEMP  dated  18  August  1989. 
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TABLE  II 


TFCC  CHRONOLOGY 


Project  Name 
MBPS 

OUTLAW  SHARK 
PCC/TPCC  Ph  I 
IPCC/ITPCC  Ph  I 
TFCC 

TPCC/FDDS* 
TFCC/FDDS  (OPEVAL) 
TPCC/FDDS** 


DT&E  Date(s) 

May-Jun  74 
Jeui-Mar  75 
Aug -Sep  75 
Jun  76  -  Jul  77 
Oct  80 
Sep  81 
Mar -May  84 
Apr  86  -  Jun  87 


OT&E  Date(s) 


Oct-Nov  80 
Sep  81 
May-Jun  84 
Jul  86-  May  87 


NOTES : 

DT&E  data  extracted  from  TFCC  TEMP  240-2  III -2  dated  10  Aug  1989. 
OT&E  data  extracted  from  NTCS-A  TEMP  1376  Annex  D  dated  13  Nov 
1992. 

*  DT  and  OT  were  combined  to  take  advantage  of  economy  of 
resources . 

**  Incremental  testing  was  performed  to  correct  discrepancies 
identified  during  OPEVAL. 
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TABLE  III 


THRESHOLDS 


Technical  Characteristics  FDDS 


JOTS  I 


System  initialization 
System  recovery 
Track  record  capacity 
Ambiguity  record  capacity 
Hard  copy 

Suitability  Qb-iectives 

Maintainability 

MTTR 

MTTR-O 

Relied}ility 

MTBMCF  (HW) 

MTBMCF  (SW) 

MTBF  (HW) 
Availability 
K 


3  min 
3  min 
500 
200 

40  sec 
£DD& 


1  Hr 
4  min 

720  Hrs 
4  Hrs 
72  Hrs 

.98 


<25  min 
<  5  min 
1000 
250 

60  sec 
JOTS  1 


1.5  Hrs 
1.0  Hr 

500  Hrs 
15  Hrs 
250  Hrs 

.90 


NOTES : 

Data  for  Table  II  were  extracted  from  JOTS  TEMP  240-10  dated  lO 
Aug  1989  and  Operational  Evaluation  of  TFCC  report  dated  07  Jan 
1985. 

MTTR  -  Mean  Time  to  Repair 

MTTR-O  -  Mean  Time  to  Restore-Operator 

MTBMCF  -  Mean  Time  Between  Mission  Critical  Failure 

A(,=  Uptime  /  Uptime  +  Downtime  (MTTR  +  Mean  Logistic  Down  Time) 
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TABLE  IV 


TEST  OBJECTIVES 


Effectiveness  Obiectives 


Results 

FDDS  JOTS 


OTC/BG  Cdr  support  * 

C2  Effectiveness  * 
Information  Exchange  ** 
Afloat  correlation  *** 
Data  Base  Management 
Tactical  Decision  Aids 
Security 


UNSAT 

UNSAT 

UNSAT 

UNSAT 

UNSAT 

UNSAT 

UNSAT 

UNSAT 

UNSAT 

SAT 

N/A 

SAT 

N/A 

UNSAT 

Suitability  Ob-iectives 

Maintainability 

Reliability 

Availability 

Logistic  Support 

Coinpatibility 

Interoperability 

Training 

Human  factors 

Safety 

Documentation 

Vulnerability 

Survivability 


Unresolved 

SAT 

SAT 

SAT 

UNSAT 

SAT 

SAT 

UNSAT 

SAT 

UNSAT 

SAT 

UNSAT 

UNSAT 

UNSAT 

UNSAT 

UNSAT 

UNSAT 

UNSAT 

UNSAT 

UNSAT 

SAT 

N/A 

SAT 

SAT 

NOTES : 

FDDS  results  were  extracted  from  NTCS-A  TEMP  1376. 

JOTS  results  were  extracted  from  JOTS  OT&E  report  19  Feb  1991. 
JOTS  equivalent  was  called  the  following: 

*  Afloat  Cdr  Support 

**  Communications  Information  Processing 
***  Attribute  Matching 
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INTEGRATED  SCHEDULE 


Source:  NTCS-A  TEMP  1376  dated  13  Nov  92. 

Reproduced  with  the  perBission  of 
„  CNO  (OP  942). 
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